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MODELING  THE  DIRECT  AND  INDIRECT  DETERMINANTS  OF  DIFFERENT  TYPES  OF 
INDIVIDUAL  JOB  PERFORMANCE 

EXECUTIVE  SUMMARY 


Research  Requirement: 

This  report  summarizes  research  carried  out  pursuant  to  Contract  #  W74V8H-05-K.-0005 
of  the  United  States  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI), 
under  the  auspices  of  its  Basic  Research  Unit.  Organizational  citizenship  performance  and 
adaptive  performance  are  important  components  of  the  Army  junior  commissioned  officer  job 
performance  construct  domain.  Despite  the  importance  of  these  constructs,  there  has  been  very 
little  research  studying  them  in  a  military  setting.  There  also  is  no  well-established  model  of  the 
process  by  which  individual  difference  variables  predict  citizenship  performance  or  adaptive 
performance,  in  either  a  military  or  a  civilian  context.  The  purpose  of  this  research  was  to  test  a 
model  of  the  process  through  which  individual  difference  variables  work  to  influence 
performance  on  specific  citizenship  and  adaptive  performance  dimensions  relevant  to  the 
military. 

Procedure: 

In  order  to  test  this  model,  we  assembled  and  developed  a  battery  of  instruments  that  are 
construct-valid  measures  of  each  component  of  the  model.  All  measures  created  for  this  project 
were  based  on  input  from  ROTC  cadets  and  midshipmen.  To  measure  task,  citizenship,  and 
adaptive  performance,  we  created  a  multisource  performance  rating  instrument  measuring 
performance  on  five  different  dimensions.  We  created  a  past  behavior  record  to  measure  skill 
and  a  situational  judgment  test  to  measure  knowledge,  both  of  which  are  relevant  to  five 
performance  dimensions.  We  also  created  (a)  a  unique  self-report  measure  of  work  habits;  (b)  a 
motives  scale  assessing  attitudes,  values,  and  preferences  that  predict  performance;  (c)  a 
motivation  scale  measuring  self-efficacy,  expectancy,  and  goal  commitment;  and  (d)  a  scale 
measuring  experience.  In  addition,  we  created  a  self-regulation  scale  to  measure  the  extent  to 
which  respondents  were  able  to  focus  on  performing  the  behaviors  they  indicated  they  intended 
to  perform,  and  assembled  a  battery  of  existing  instruments  to  measure  cognitive  ability, 
personality,  experience,  social  insight,  and  action  control. 

The  predictor  battery  was  administered  to  155  ROTC  cadets  and  midshipmen  in  the  Fall 
of  2006.  Criterion  administration  took  place  during  March  and  April  of  2007,  allowing  us  to 
assess  the  extent  to  which  cadets  and  midshipmen  performed  the  behaviors  they  indicated  they 
were  motivated  to  perform.  The  self-regulation  scale  was  administered  four  times  during  this 
period,  allowing  us  to  track  how  well  each  cadet/midshipman  was  able  to  maintain  focus  on 
performing  these  behaviors.  Each  cadet/midshipman  was  rated  by  one  to  five  peers.  The  data 
were  analyzed  using  structural  equation  modeling  to  test  the  Johnson  (2003)  model,  as  well  as 
several  ancillary  hypotheses. 
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Findings: 


The  data  summarized  in  this  report  show  that  modified  versions  of  the  general  Johnson 
(2003)  model  were  found  to  describe  the  process  by  which  predictor  variables  influence 
performance.  Different  models  were  necessary  to  describe  the  process  for  task,  citizenship,  and 
adaptive  performance.  The  strongest  support  for  the  model  came  from  the  task  performance 
dimension.  For  this  dimension,  knowledge,  skill,  cognitive  choice  aspects  of  motivation,  and 
self-regulation  were  all  direct  determinants  of  performance,  and  each  of  these  determinants  was 
predicted  by  different  combinations  of  personality  traits,  experience,  and  motives.  Motives  such 
as  job  satisfaction  and  affective  commitment  were  related  to  performance  only  through  the 
mediating  influence  of  expectancies  and  goal  commitment.  A  similar  model  was  supported  for 
the  adaptive  performance  dimension.  The  differences  were  that  only  some  of  the  hypothesized 
direct  determinants  had  significant  paths  to  performance,  and  one  personality  trait  had  a  direct 
influence  on  performance. 

This  model  did  not  generalize  to  the  citizenship  performance  dimensions,  although  a 
similar  model  was  found  for  each  different  dimension  of  citizenship  performance.  In  this  model, 
motives  were  a  direct  determinant  of  performance  and  expectancies  were  not.  Knowledge,  skill, 
and  self-regulation  were  direct  determinants,  but  never  all  at  once  and  the  one  that  was  a 
determinant  depended  on  the  citizenship  dimension  of  interest.  There  were  two  cases  in  which  a 
hypothesized  indirect  performance  determinant  had  a  direct  influence  on  performance. 

Utilization  and  Dissemination  of  Findings: 

Several  potential  applications  of  this  basic  research  were  described.  First,  this  model 
contributes  to  a  better  understanding  of  the  relationships  between  predictors,  mediators,  and  job 
performance  criteria,  and  could  be  used  to  choose  appropriate  predictors  for  whatever  criterion 
construct  is  of  interest  for  a  particular  Army  job.  This  model  of  the  influence  of  individual 
differences  on  job  performance  can  not  only  facilitate  selection,  classification,  and  training  of 
junior  commissioned  Army  officers,  but  also  adds  significantly  to  the  evolving  understanding  of 
job  performance.  The  Army  can  use  this  model  to  identify  interventions  that  will  have  the 
greatest  impact  on  areas  of  performance  that  are  deficient  in  certain  officers.  This  tool  also  would 
be  effective  in  identifying  training  and/or  development  needs.  Given  a  criterion  construct  on 
which  an  individual’s  performance  is  in  need  of  improvement,  this  model  can  help  to  identify  the 
determinants  of  performance  on  that  construct.  The  products  developed  for  this  project  also  may 
be  useful  as  self-development,  training,  selection,  or  feedback  tools. 
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Background 


To  make  sense  of  the  host  of  potential  job  performance  determinants  (i.e.,  predictors  of 
performance)  from  which  to  choose  when  doing  performance  prediction  research,  a  model  of  the 
process  by  which  specific  determinants  influence  performance  on  specific  dimensions  is 
necessary.  Research  on  models  of  performance  determinants  has  been  conducted  in  both  military 
and  civilian  contexts,  but  has  been  primarily  focused  on  the  relationship  between  cognitive 
ability  and  task  performance  (Borman,  White,  Pulakos,  &  Oppler,  1991;  Hunter,  1983;  Lance  & 
Bennett,  2000;  Schmidt,  Hunter,  &  Outerbridge,  1986).  Task  performance  consists  of  activities 
that  (a)  directly  transform  raw  materials  into  the  goods  and  services  produced  by  the 
organization,  or  (b)  service  and  maintain  the  technical  core  by  replenishing  supplies;  distributing 
products;  and  providing  planning,  coordination,  supervising,  and  staff  functions  that  allow  for 
efficient  functioning  of  the  organization  (Borman  &  Motowidlo,  1993). 

Citizenship  Performance 

Task  performance  can  be  distinguished  from  contextual  performance,  which  is  defined  as 
activities  that  support  the  broader  environment  in  which  the  technical  core  must  function, 
including  behaviors  such  as  volunteering  for  tasks  not  formally  part  of  the  job,  demonstrating 
effort,  helping  and  cooperating  with  others,  following  organizational  rules  and  procedures,  and 
supporting  organizational  objectives  (Borman  &  Motowidlo,  1993).  Contextual  performance, 
organizational  citizenship  behavior,  and  related  concepts  are  often  referred  to  under  the  general 
label  citizenship  performance  (Borman  &  Penner,  2001;  Coleman  &  Borman,  2000). 
Confirmatory  factor  analyses  have  provided  evidence  for  the  distinction  between  task  and 
citizenship  performance  (Conway,  1996;  Johnson,  2001).  Further,  research  has  shown  that  both 
task  performance  and  citizenship  performance  are  taken  into  consideration  when  supervisors 
evaluate  others’  performance  (Conway,  1999;  Johnson,  2001;  Motowidlo  &  Van  Scotter,  1994; 
Van  Scotter  &  Motowidlo,  1996).  Some  research  also  shows  that  task  performance  is  better 
predicted  by  ability  and  experience,  and  citizenship  performance  is  better  predicted  by 
personality  variables  (Borman,  Penner,  Allen,  &  Motowidlo,  2001;  Hurtz  &  Donovan,  2000). 

The  dimensionality  of  citizenship  performance  is  muddled,  with  different  authors 
offering  different  numbers  of  dimensions  with  different  labels  (Borman  &  Motowidlo,  1993; 
LePine,  Erez,  &  Johnson,  2002;  Organ,  1988;  Podsakoff,  MacKenzie,  Paine,  &  Bachrach,  2000). 
In  an  attempt  to  clarify  the  latent  structure  of  citizenship  performance,  Coleman  and  Borman 
(2000)  identified  27  citizenship  performance  behaviors  based  on  all  proposed  models  and 
discussions  presented  in  the  literature.  The  behaviors  were  sorted  by  44  industrial  and 
organizational  (IO)  psychologists,  and  the  similarity  data  were  analyzed  using  factor  analysis, 
multidimensional  scaling,  and  cluster  analysis.  The  authors  rationally  combined  the  results  of  the 
separate  analyses  into  a  single  integrated  model  representing  three  categories  of  behavior. 
Borman,  Buck,  Hanson,  Motowidlo,  Stark,  and  Drasgow  (2001)  refined  this  taxonomy  on  the 
basis  of  a  sort  of  approximately  2,300  examples  of  citizenship  performance  taken  from  22 
studies,  giving  the  categories  the  following  labels:  (a)  personal  support,  (b)  organizational 
support,  and  (c)  conscientious  initiative.  Personal  support  consists  of  behaviors  benefiting 
individuals  in  the  organization,  and  includes  helping,  motivating,  cooperating  with,  and  showing 
consideration  of  others.  Organizational  support  consists  of  behaviors  benefiting  the  organization. 
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and  includes  representing  the  organization  favorably,  showing  loyalty,  and  complying  with 
organizational  rules  and  procedures.  Conscientious  initiative  consists  of  behaviors  benefiting  the 
job  or  task,  and  includes  persisting  with  extra  effort  to  complete  tasks,  taking  initiative,  and 
engaging  in  self-development  activities  (Borman,  Penner,  et  al.,  2001). 

Research  in  military  settings  showing  the  importance  of  citizenship  performance  to  the 
criterion  domain  has  mostly  been  limited  to  enlisted  personnel  (Borman,  White,  &  Dorsey,  1995; 
Campbell,  Hanson,  &  Oppler,  2001;  Campbell,  McHenry,  &  Wise,  1990;  Motowidlo  &  Van 
Scotter,  1 994).  Given  the  importance  of  citizenship  performance  to  military  enlisted  jobs  and  all 
types  of  civilian  jobs,  it  is  surprising  that  there  is  so  little  published  research  on  citizenship 
performance  in  military  officer  jobs  (two  dissertations  have  examined  citizenship  performance 
among  Army  officers  or  West  Point  cadets;  Grojean,  2002;  Leboeuf,  1995).  Geraghty  and 
Collins  (2003)  noted  that  military  officer  jobs  involve  citizenship  behaviors  that  are  relevant  to 
almost  every  job  (e.g.,  altruism,  courtesy,  personal  discipline),  as  well  as  supervisory  citizenship 
behaviors  such  as  motivating  and  supporting  others. 

The  Army’s  mission  documents  (e.g..  Department  of  the  Army,  1986,  cited  in  Bartone  & 
Kirkland,  1991)  emphasize  the  importance  of  unit  cohesion  to  military  combat  readiness  and  the 
importance  of  sound  leadership  exercised  by  junior  officers  to  the  development  of  cohesion  in 
these  units.  Group  cohesion  (i.e.,  the  tendency  for  a  group  to  stick  together  and  remain  united  in 
the  pursuit  of  its  goals;  Carron,  1982)  in  military  units  has  been  researched  extensively.  In  a 
large-scale  meta-analysis  of  military  group  cohesion  research,  Oliver,  Harman,  Hoover,  Mayes, 
and  Pandhi  (1999)  found  positive  relationships  between  cohesion  and  group  performance, 
individual  performance,  job/military  satisfaction,  retention,  well-being,  and  readiness;  and  a 
negative  relationship  between  cohesion  and  disciplinary  problems. 

In  Bartone  and  Kirkland’s  (1991)  model  of  the  development  of  excellent  military  units, 
the  decisive  factor  differentiating  cohesive,  high-performance  units  from  mediocre  units  is  the 
behavior  of  leaders.  The  authors  identified  four  critical  leader  characteristics  that  contribute  to 
the  development  of  excellent  units  at  four  different  stages.  The  behaviors  associated  with  three  of 
these  characteristics  represent  different  aspects  of  citizenship  performance  (e.g.,  show  concern 
for  Soldier  well-being,  treat  Soldiers  with  dignity,  and  share  Soldiers’  sense  of  mission).  A 
longitudinal  study  of  an  Army  medical  task  force  found  support  for  this  position  (Bartone  & 

Adler,  1999).  In  that  study,  one  of  the  most  important  correlates  of  task  force  cohesion 
throughout  deployment  was  the  Soldiers’  perceptions  that  their  leaders  were  concerned  about 
them  (an  aspect  of  personal  support). 


Adaptive  Performance 

Task  and  citizenship  performance  have  been  generally  accepted  as  separate  aspects  of  the 
individual  job  performance  domain.  Because  of  the  increasingly  dynamic  nature  of  work 
environments,  however,  adaptive  performance  has  recently  received  increased  attention 
(Campbell,  1999;  Hesketh  &  Neal,  1999;  London  &  Mone,  1999;  Pulakos,  Arad,  Donovan,  & 
Plamondon,  2000).  Adaptive  performance  is  the  proficiency  with  which  a  person  alters  his  or  her 
behavior  to  meet  the  demands  of  the  environment,  an  event,  or  a  new  situation  (Pulakos  et  al., 
2000).  Hesketh  and  Neal  (1999),  Schmitt,  Cortina,  Ingerick,  and  Wiechmann  (2003),  and 
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Johnson  (2003)  suggested  that  adaptive  performance  is  a  component  of  the  performance  domain 
that  is  separate  from  task  and  citizenship  performance.  Pulakos  et  al.  (2000)  developed  and  found 
support  for  a  taxonomy  of  adaptive  performance  consisting  of  eight  dimensions.  Johnson  (2001 , 
2003)  classified  seven  of  these  dimensions  as  either  task  performance,  citizenship  performance, 
or  a  combination  of  both.  The  dimension  of  dealing  with  uncertain  and  unpredictable  work 
situations  is  the  only  component  of  Pulakos  et  al.’s  taxonomy  that  is  clearly  distinct  from  task 
and  citizenship  performance  (Johnson,  2003).  Elements  of  this  dimension  include  taking  action 
when  necessary  without  having  all  the  facts  at  hand;  adjusting  plans,  actions,  or  priorities  to  deal 
with  changing  situations;  and  imposing  structure  to  provide  focus  in  dynamic  situations. 

Adaptive  performance  is  undoubtedly  an  important  component  of  performance  for  many 
Army  officer  jobs.  For  example,  officers  and  NCOs  must  frequently  adapt  to  changes,  including 
deployment  in  other  countries,  where  they  must  provide  leadership  in  newly  formed,  specialized 
task  forces  (Bartone  &  Adler,  1999).  Pulakos  et  al.  (2000)  found  that  the  dimension  of  dealing 
with  uncertain  and  unpredictable  work  situations  was  among  the  most  important  dimensions  to 
Special  Forces  Soldiers,  combat  NCOs,  and  military  police. 

Thus,  there  is  evidence  that  citizenship  performance  and  adaptive  performance  are 
important  components  of  the  Army  officer  job  performance  construct  domain,  just  as  they  are  in 
most  other  military  and  civilian  jobs.  However,  there  has  been  very  little  research  studying  these 
performance  components  in  a  military  setting.  There  is  also  no  well-established  model  of  the 
process  by  which  individual-difference  variables  predict  citizenship  performance  or  adaptive 
performance,  in  either  a  military  or  a  civilian  context.  The  purpose  of  this  project  was  to  test  a 
model  of  the  process  through  which  individual  difference  variables  work  to  influence 
performance  on  specific  citizenship  and  adaptive  performance  dimensions  relevant  to  officers  in 
the  U.S.  Army. 


Models  of  Individual  Differences  in  Job  Performance 

Several  models  of  how  individual  differences  influence  job  performance  have  been 
proposed.  According  to  Campbell,  McCloy,  Oppler,  and  Sager  (1993),  performance  is  a  function 
of  three  determinants:  (a)  declarative  knowledge  (i.e.,  factual  knowledge  about  specific  things), 

(b)  procedural  knowledge/skill  (i.e.,  degree  to  which  one  is  actually  able  to  perform  a  task),  and 

(c)  motivation  (i.e.,  the  combined  effect  of  the  choice  to  expend  effort,  the  choice  of  the  level  of 
effort  to  expend,  and  the  choice  to  persist  at  that  level  of  effort).  Performance  on  a  given  job 
dimension  is  determined  directly  by  some  combination  of  these  three  determinants.  The  direct 
determinants  are  distinguished  from  indirect  performance  determinants,  which  can  only 
influence  performance  via  the  direct  determinants.  Examples  of  indirect  determinants  provided 
by  the  organization  include  reward  systems,  training,  and  management  practices.  Examples  of 
indirect  determinants  that  the  individual  brings  to  the  organization  are  abilities,  personality, 
education,  and  experience. 

Campbell  et  al.’s  (1993)  model  of  performance  determinants  provides  a  general 
explanation  for  how  individual  differences  in  predictor  variables  translate  to  individual 
differences  in  job  performance  on  a  particular  dimension.  Motowidlo,  Borman,  and  Schmit 
(1997)  expanded  the  Campbell  et  al.  model  to  explain  why  personality  should  be  a  better 
predictor  of  citizenship  performance  dimensions  than  of  task  performance  dimensions.  They  split 
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declarative  knowledge  and  procedural  knowledge/skill  into  task  knowledge  and  skill  and 
citizenship  knowledge  and  skill.  Task  knowledge  is  knowledge  of  facts,  principles,  and 
procedures  relevant  to  the  core  technical  functions  of  the  organization;  task  skill  is  skill  in 
performing  necessary  actions  to  complete  tasks.  Citizenship  knowledge  is  knowledge  of  facts, 
principles,  and  procedures  relevant  to  maintaining  the  organizational  environment  in  which  the 
technical  core  must  function  (e.g.,  knowing  how  to  cooperate  with  others,  knowing  how  to 
present  a  favorable  image  of  the  organization).  Contextual  skill  is  skill  in  performing  actions 
known  to  be  effective  in  situations  calling  for  citizenship  performance.  Task  knowledge  and  skill 
are  determined  primarily  by  cognitive  ability,  which  is  supported  by  ample  research  (Borman  et 
al.,  1995;  Hunter,  1983;  Lance  &  Bennett,  2000).  Motowidlo  et  al.  suggested  that  personality 
should  be  the  primary  determinant  of  citizenship  knowledge  and  skill,  because  people  possessing 
personality  characteristics  consistent  with  a  particular  element  of  citizenship  knowledge  or  skill 
should  be  more  likely  to  notice  the  relative  effectiveness  of  certain  patterns  of  behavior  in 
relevant  situations,  and  thus  more  likely  to  master  that  knowledge  or  skill. 

The  Motowidlo  et  al.  (1997)  model  is  further  distinguished  from  the  Campbell  et  al. 
(1993)  model  by  replacing  motivation  with  task  and  citizenship  work  habits.  Work  habits  are 
patterns  of  behavior  people  learn  over  time  that  can  facilitate  or  interfere  with  job  performance. 
Work  habits  include  characteristic  motivational  responses  such  as  choices  for  the  amount, 
intensity,  and  duration  of  effort  to  expend;  tendencies  to  approach  or  avoid  certain  situations; 
procrastination;  or  persistence  in  the  face  of  adversity.  Work  habits  also  include  characteristic 
responses  that  are  not  necessarily  motivational  in  nature  (e.g.,  an  officer  who  has  been  trained  in 
the  best  way  to  deal  with  a  problem  subordinate,  but  occasionally  reverts  to  pre-training  habits  of 
reacting  with  hostility).  Task  work  habits  are  characteristic  responses  to  situations  that  interfere 
with  or  facilitate  the  completion  of  tasks.  Citizenship  work  habits  are  characteristic  responses 
that  interfere  with  or  facilitate  performance  in  citizenship  work  situations.  Motowidlo  et  al. 
suggested  that  task  habits  are  predicted  by  both  cognitive  ability  and  certain  personality  variables 
(e.g.,  conscientiousness),  and  citizenship  habits  are  predicted  primarily  by  certain  other 
personality  variables  (e.g.,  agreeableness,  extroversion).  Because  personality  variables  are 
expected  to  influence  more  determinants  on  the  citizenship  side  of  the  model  and  ability 
variables  are  expected  to  influence  more  determinants  on  the  task  side  of  the  model,  personality 
should  be  more  related  to  citizenship  performance  and  ability  should  be  more  related  to  task 
performance. 


A  More  Complete  Model  of  Performance  Determinants 

The  Campbell  et  al.  (1993)  and  Motowidlo  et  al.  (1997)  models  are  good  general  models 
of  performance  prediction,  but  they  are  deficient  in  that  they  do  not  explain  the  process  by  which 
specific  individual  differences  influence  performance  on  specific  dimensions.  Johnson  (2003) 
pointed  out  that  motivation  is  given  inadequate  attention  by  these  models  and  proposed  an 
expanded  model  of  how  individual  differences  influence  job  performance.  This  model  is 
compatible  with  those  of  Campbell  et  al.  and  Motowidlo  et  al.,  but  adds  elements  to  both.  The 
major  difference  is  an  expanded  conceptualization  of  motivation.  Campbell  et  al.  use  a  cognitive 
choice  model  of  motivation,  in  which  the  choice  to  perform  leads  directly  to  behavior.  There  is 
no  explicit  provision  for  motivational  processes  that  may  be  used  to  overcome  difficulties  in  the 
accomplishment  of  the  intention  to  perform. 


4 


Motowidlo  et  al.  replaced  motivation  with  work  habits,  which  they  defined  as  stylistic 
ways  people  handle  different  kinds  of  situations  that  occur  on  the  job,  learned  as  their  basic 
tendencies  (personality  traits)  interact  with  their  environments  over  time.  Habits  are  an  important 
component  to  include  in  a  model  of  performance  determinants  because  they  may  interfere  with 
performance  despite  motivation  to  perform  in  a  certain  way.  Rather  than  replacing  the  motivation 
component,  however,  Johnson  (2003)  included  work  habits  in  addition  to  motivation.  Although 
Motowidlo  et  al.  included  choices  for  how  much  effort  to  exert  and  for  how  long  as  examples  of 
characteristic  motivational  responses  under  work  habits,  this  appears  to  exclude  motivational 
choices  that  go  against  one’s  habitual  tendencies.  For  example,  a  person’s  characteristic  tendency 
may  be  to  exert  as  little  effort  as  possible,  but  he  or  she  may  choose  to  go  against  that  tendency 
in  response  to  a  new  bonus  structure  that  rewards  productivity. 

Because  motivation  is  such  an  important  mediating  variable  between  individual 
differences  and  job  performance,  Johnson  (2003)  more  completely  described  that  aspect  of  the 
model  in  an  attempt  to  better  understand  the  nature  of  these  relationships.  Johnson  noted  that 
habits  influence  behavior,  despite  intentions  to  behave  otherwise,  because  they  require  very  little 
attention.  To  implement  an  intention  that  goes  against  habitual  tendencies  and  other  intentions 
competing  for  one’s  attention,  one  must  engage  self-regulatory  or  volitional  mechanisms.  Self¬ 
regulation  refers  to  the  higher-level  cognitive  processes  that  guide  the  allocation  of  attention, 
time,  and  effort  across  activities  directed  toward  attaining  a  goal  (Kanfer,  1990)  and  protect  an 
intention  from  being  replaced  by  a  competing  action  tendency  before  the  intended  action  is 
completed  (Kuhl,  1985).  Self-regulation  is  a  critical  component  of  motivation  that  is  missing 
from  the  Campbell  et  al.  (1993)  and  Motowidlo  et  al.  (1997)  models. 

Some  theories  integrate  cognitive  choice  and  self-regulatory  aspects  of  motivation 
(Heckhausen  &  Kuhl,  1985).  The  importance  of  this  integrative  perspective  is  that  different 
dispositional  variables  are  proposed  to  influence  motivation  at  different  stages,  providing  a 
framework  for  more  systematic  investigation  of  how  individual  differences  (primarily 
personality)  affect  motivation  and  job  performance  (Kanfer,  1990).  Mitchell  and  Daniels  (2003) 
identified  two  components  of  motivation:  (a)  proactive  cognitive  processes,  and  (b)  on-line 
cognitive  processes.  Proactive  cognitive  processes  occur  before  a  task  is  commenced,  and  reflect 
cognitions  about  expectations  for  achieving  a  goal  or  the  value  of  outcomes  resulting  from 
achieving  a  goal.  During  this  phase,  people  determine  what  course  of  action  to  take,  resulting  in 
the  formation  of  an  intention.  Mitchell  and  Daniels  include  expectancy,  self-efficacy,  and  goal¬ 
setting  in  the  proactive  category  of  motivation  theories.  On-line  cognitive  processes  occur  while 
the  person  is  working  on  a  task,  and  are  characterized  by  self-regulatory  processes  that  are 
necessary  to  maintain  goal-directed  action.  This  phase  refers  to  the  process  of  implementing  an 
intention  to  achieve  a  goal.  Control  theory,  action  theory,  and  self-regulation  are  on-line  theories 
of  motivation  (Mitchell  &  Daniels,  2003). 

Johnson  (2003)  added  psychological  motives  as  a  third  component  of  motivation.  A 
motive  is  a  reason  (e.g.,  value,  interest,  preference,  attitude)  for  choosing  to  exert  effort  in  a 
particular  direction.  Motive-based  theories  recognize  that  people  may  have  very  different 
purposes  for  exhibiting  the  same  behavior  (Borman  &  Penner,  2001).  For  example,  one  officer 
may  help  another  complete  a  task  because  the  officer  values  helping  others  succeed.  Another 
officer  may  exhibit  the  same  helping  behavior  because  he  or  she  believes  it  will  benefit  the  Army 
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to  get  the  work  done.  The  first  officer  has  an  altruistic  motive  and  the  second  officer  has  an 
organizational  commitment  motive,  but  they  both  lead  to  the  same  behavior.  Motives  are 
expected  to  directly  influence  proactive  cognitive  processes  (Johnson,  2003;  Kanfer,  1992). 

One  type  of  motive  that  has  been  extensively  studied  is  job  attitudes.  Job  attitudes  tend  to 
be  more  strongly  related  to  citizenship  performance  than  are  personality  variables  (Podsakoff  et 
al.,  2000),  leading  Organ  and  Ryan  (1995)  to  conclude  that  the  relationship  between  personality 
and  citizenship  performance  is  probably  mediated  by  attitudes  such  as  job  satisfaction, 
organizational  commitment,  and  fairness  perceptions.  Organizational  commitment  is  an 
especially  important  motive  for  the  military  context  (Gade,  2003),  and  its  components  have  been 
shown  to  predict  various  types  of  job-related  behaviors  (Gade,  Tiggle,  &  Schumm,  2003; 
Karrasch,  2003).  Given  that  affective  commitment  to  the  organization  is  related  to  the  extent  to 
which  individuals  engage  in  citizenship  behaviors  (Allen  &  Meyer,  1996),  this  is  an  important 
motive  to  evaluate  for  military  officers. 

Johnson’s  (2003)  general  model  of  the  pathways  by  which  individual  differences  in 
predictor  variables  influence  performance  on  a  given  dimension  is  presented  in  Figure  1 . 
Consistent  with  Campbell  et  al.  (1993),  performance  is  a  function  of  knowledge,  skill,  and 
motivation.  The  Johnson  model  expands  motivation  into  the  components  of  motives,  proactive 
cognitive  processes,  and  on-line  cognitive  processes.  Besides  the  expanded  conceptualization  of 
motivation,  another  difference  from  Campbell  et  al.  is  the  addition  of  work  habits  as  a  fourth 
determinant  in  recognition  of  the  possibility  that  job-relevant  behavior  can  occur  automatically 
despite  motivation  to  behave  otherwise.  Work  habits  also  influence  performance  indirectly  by 
influencing  the  need  for  and  choice  of  self-regulatory  strategies.  For  simplicity,  the  only  indirect 
determinants  included  are  personality  variables  and  ability  variables.  This  model  could  be 
expanded  to  include  other  classes  of  individual  (experience,  interests)  or  organizational  (training, 
rewards)  indirect  performance  determinants.  The  model  also  recognizes  the  numerous  potential 
moderators  (e.g.,  occupation,  tenure)  that  can  influence  the  extent  to  which  individual  differences 
predict  performance. 
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Knowledge 


r- 


individual  differences  on  specific  dimensions  of  job  performance. 


The  model  in  Figure  1  is  a  general  model  describing  the  potential  paths  through  which 
different  classes  of  variables  may  operate  to  influence  different  types  of  performance.  The 
relative  strength  of  each  path  from  one  construct  to  another  depends  on  the  specific  predictor 
variables  included  in  the  model  and  the  specific  performance  dimension  that  is  the  criterion.  For 
example,  if  achievement  were  used  to  predict  the  demonstrating  effort  dimension  of  citizenship 
performance,  the  strongest  path  would  likely  go  through  motivation  because  motivation  is  highly 
relevant  to  demonstrating  effort,  and  achievement  is  highly  relevant  to  each  component  of 
motivation.  If  sociability  were  used  to  predict  the  maintaining  good  working  relationships 
dimension  of  citizenship  performance,  however,  the  stronger  paths  would  likely  go  through 
knowledge  and  skill.  This  is  because  social  knowledge  and  skill  are  highly  relevant  to 
maintaining  good  working  relationships,  they  are  likely  to  be  predicted  by  sociability,  and 
sociability  is  not  as  strong  a  predictor  of  motivation. 

The  defining  feature  of  the  model  that  should  be  consistent  across  all  predictors  and 
criteria  is  that  performance  is  directly  determined  by  one  or  more  of  the  following  constructs:  (a) 
knowledge,  (b)  skill,  (c)  motivation,  and  (d)  habits.  Indirect  performance  determinants  such  as 
personality,  ability,  and  experience  should  only  influence  performance  through  the  direct 
determinants.  Performance  on  many  dimensions  may  be  a  function  of  all  four  determinants, 
while  performance  on  other  dimensions  may  be  a  function  of  only  one,  two,  or  three.  For 
example,  habits  may  not  predict  performance  on  a  dimension  that  requires  behavior  that  is  not 
amenable  to  automatic  processing.  Other  performance  dimensions,  particularly  some  in  the 
citizenship  performance  domain,  may  require  very  little  knowledge  or  skill  and  are  determined 
almost  entirely  by  motivational  processes  (e.g.,  volunteering  for  extra  work). 

The  purpose  of  this  model  is  to  identify  the  constructs  through  which  individual 
difference  variables  work  to  influence  performance  on  specific  performance  dimensions.  It  can 
be  used  to  choose  appropriate  predictors  for  a  given  criterion  construct.  The  strength  of  the 
relationship  between  the  predictor  and  the  criterion  depends  on  (a)  the  number  of  direct 
determinants  of  the  criterion  to  which  the  predictor  is  related,  (b)  the  strength  of  the  relationship 
between  the  predictor  and  each  direct  determinant,  (c)  the  strength  of  the  relationship  between 
each  direct  determinant  and  the  criterion,  and  (d)  the  presence  of  relevant  moderators. 

When  predicting  task  performance  dimensions,  ability  variables  should  be  most 
predictive  because  of  their  strong  relationships  with  task  knowledge,  task  skill,  and  task  habits, 
and  the  strong  relationships  between  these  direct  determinants  and  task  performance.  Personality 
variables  also  should  contribute,  but  to  a  lesser  degree,  because  of  their  strong  relationships  with 
motivation  but  weaker  relationships  with  task  knowledge,  task  skill,  and  task  habits.  When 
predicting  citizenship  performance,  personality  variables  should  be  most  predictive  because  of 
their  strong  relationships  with  motivation  and  citizenship  knowledge,  skill,  and  habits.  Ability 
variables  should  be  predictive  to  a  lesser  extent  because  of  their  weaker  associations  with  these 
constructs.  Personality  variables  are  probably  most  predictive  of  adaptive  performance  because 
of  the  importance  of  self-regulatory  skills  when  quickly  adjusting  to  a  new  situation.  Ability 
variables  should  also  be  strongly  related  because  of  the  importance  of  skills  such  as  problem 
solving. 
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Overview  of  Research 


The  purpose  of  this  research  was  to  test  the  Johnson  (2003)  model  of  performance 
determinants  for  different  kinds  of  performance,  explicating  the  relative  strength  of  different 
paths  for  specific  performance  dimensions.  To  accomplish  this,  we  developed  or  obtained 
measures  of  each  construct  specified  in  the  model,  including  a  large  number  of  potential  indirect 
performance  determinants  (e.g.,  cognitive  ability,  personality  scales,  action  control  scales,  and 
attitudes).  We  then  administered  each  instrument  to  a  sample  of  ROTC  cadets,  each  of  whom 
was  rated  on  their  performance  by  one  to  four  peers.  Participants  were  third-  and  fourth- year 
cadets  with  experience  leading  others  to  enhance  the  generalizability  of  the  results  to  junior 
officers  in  the  Armed  Forces.  This  allowed  us  to  obtain  data  from  a  population  that  was 
accessible  to  us,  and  enabled  us  to  examine  constructs  and  develop  instruments  that  are  relevant 
to  the  military.  Each  cadet  was  rated  on  (a)  decision  making/problem  solving,  (b)  maintaining 
good  working  relationships,  (c)  organizational  commitment,  (d)  showing  initiative,  and  (e) 
adapting  to  uncertain  or  changing  situations.  The  first  dimension  is  an  element  of  task 
performance  (Johnson,  2003),  the  middle  three  dimensions  all  tap  different  aspects  of  citizenship 
performance  (Borman,  Buck  et  al.,  2001),  and  the  final  dimension  is  an  element  of  adaptive 
performance  (Pulakos  et  al.,  2000). 

To  demonstrate  that  each  performance  dimension  is  an  important  part  of  the  performance 
domain  for  the  population  of  interest,  we  determined  the  unique  and  relative  contribution  of  each 
dimension  to  ratings  of  overall  job  performance.  We  expected  that  each  performance  dimension 
would  contribute  unique  variance  to  ratings  of  overall  performance.  In  addition  to  identifying  the 
unique  variance  contributed  by  each  performance  dimension,  we  estimated  the  relative 
importance  raters  place  on  each  dimension  by  conducting  a  relative  weight  analysis  (Johnson, 
2000).  Relative  weight  analysis  is  a  procedure  for  quantifying  the  relative  importance  of 
predictor  variables  in  multiple  regression.  Relative  weights  reflect  the  proportionate  contribution 
each  predictor  makes  to  R2,  considering  both  its  unique  contribution  and  its  contribution  when 
combined  with  other  variables.  Thus,  relative  weights  are  interpretable  as  measures  of  relative 
importance  even  when  predictor  variables  are  correlated.  Johnson  (2001)  used  this  procedure  to 
demonstrate  the  relative  importance  of  specific  task  and  citizenship  performance  dimensions  to 
evaluations  of  overall  performance.  We  expected  that  each  performance  dimension  would  be 
relatively  important  to  overall  performance  judgments. 

We  used  structural  equation  modeling  to  evaluate  the  Johnson  (2003)  model  for  each  of 
the  five  performance  dimensions.  Support  for  the  model  is  indicated  by  (a)  fit  statistics  that  meet 
the  standards  for  good  model  fit,  and  (b)  significant  path  coefficients  indicating  that  the  paths 
between  indirect  performance  determinants  and  performance  are  not  direct  but  rather  go  through 
the  hypothesized  mediating  variables.  We  expected  that  approximately  the  same  model  would 
describe  the  performance  prediction  process  for  all  types  of  performance  dimensions,  in  that  the 
hypothesized  indirect  performance  determinants  would  influence  performance  only  through  the 
hypothesized  direct  performance  determinants.  The  relative  strength  of  different  paths  should 
vary  across  different  dimensions  of  performance. 

Support  for  this  model  would  provide  researchers  and  practitioners  with  a  tool  that  can  be 
used  to  identify  the  constructs  through  which  individual  difference  variables  work  to  influence 
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performance  on  specific  dimensions.  This  is  especially  important  for  identifying  predictors  of 
citizenship  and  adaptive  performance.  Such  a  tool  would  contribute  to  a  better  understanding  of 
the  relationships  between  predictors,  mediators,  and  job  performance  criteria,  and  could  be  used 
to  choose  appropriate  predictors  for  a  given  criterion  construct. 

Method 

Participants 


Examinees 

Participants  for  whom  predictor  data  and  performance  ratings  were  obtained  were  1 55 
cadets  and  midshipmen  in  Army,  Navy,  and  Air  Force  ROTC  programs  at  six  U.S.  universities 
(Colorado  State  University,  Michigan  State  University,  Purdue  University,  University  of 
Minnesota,  University  of  North  Carolina  -  Charlotte,  and  University  of  South  Florida). 
Participants  were  recruited  through  in-class  announcements  and  advertising  in  ROTC  buildings. 
Only  junior  and  senior  level  students  were  eligible  to  participate  in  the  research  to  ensure 
adequate  ROTC  experiences.  Service  branch  representation  was  54.2%  Air  Force,  35.3%  Army, 
and  1 0.5%  Navy.  The  sample  contained  23  participants  (15%)  who  had  prior  military  experience. 
Among  these  cadets/midshipmen,  the  mean  was  4.3  years  of  prior  military  service  (SD  =  2.6). 
Complete  demographic  characteristics  for  the  sample  are  presented  in  Table  1. 
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Table  1 

Demographic  Characteristics  of  Examinee  Sample 


Gender 

N 

Percentage 

Female 

30 

19 

Male 

125 

81 

Race 

White 

136 

88 

Black 

6 

4 

Hispanic 

2 

1 

Asian 

7 

4 

Other 

4 

3 

ROTC  Year 

Third 

92 

59 

Fourth 

62 

40 

Other 

1 

1 

ROTC  Service  Branch 

Army 

56 

36 

Air  Force 

83 

53 

Marine  Corps 

4 

3 

Navy 

12 

8 

University 

University  of  Minnesota 

36 

23 

University  of  South  Florida 

11 

7 

University  of  North  Carolina — Charlotte 

3 

2 

Colorado  State  University 

25 

16 

Michigan  State  University 

12 

8 

Purdue  University 

68 

44 

Raters 


Participants  who  completed  performance  ratings  were  90  cadets/midshipmen  who  were  in 
the  same  ROTC  program  as  the  people  they  rated.  Each  rater  designated  a  set  of  possible  ratees 
based  on  their  self-assessment  of  their  ability  to  accurately  rate  the  person  on  each  of  the  five 
dimensions  of  performance.  Raters  rated  an  average  of  5.6  ratees  (SD  =  2.8).  Demographic  data 
were  not  collected  from  raters,  but  because  75  raters  also  participated  in  the  earlier  portion  of  the 
project,  demographic  data  were  available  for  them.  Table  2  presents  demographic  data  for  this 
sample  of  raters. 
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Table  2 

Demographic  Characteristics  of  Rater  Sample 


Gender 

N 

Percentage 

Female 

19 

21 

Male 

56 

75 

Race 

White 

67 

89 

Black 

2 

3 

Hispanic 

0 

0 

Asian 

3 

4 

Other 

3 

4 

ROTC  Year 

Third 

43 

57 

Fourth 

32 

43 

ROTC  Service  Branch 

Army 

29 

39 

Air  Force 

38 

51 

Marine  Corps 

2 

2 

Navy 

6 

8 

University 

University  of  Minnesota 

23 

31 

University  of  South  Florida 

6 

8 

University  of  North  Carolina — Charlotte 

1 

1 

Colorado  State  University 

8 

11 

Michigan  State  University 

3 

4 

Purdue  University 

34 

45 

Note.  Demographic  data  were  missing  for  1 5  raters. 

Measures 

We  developed  or  obtained  one  or  more  measures  relevant  to  each  construct  specified  in 
the  model  (see  Figure  1 ).  To  measure  performance  components,  we  developed  a  performance 
rating  instrument  to  be  completed  by  peer  raters  that  measures  an  individual’s  level  of 
performance  on  five  different  dimensions.  In  Figure  1,  indirect  performance  determinants  were 
represented  as  ability  variables  and  personality  variables.  We  measured  general  cognitive  ability 
as  the  ability  variable  and  a  variety  of  personality  traits,  including  motivational  traits,  as  the 
personality  variables.  In  addition,  we  measured  the  following  indirect  performance  determinants: 
(a)  social  insight,  (b)  experience,  and  (c)  action  control.  We  measured  the  following  motives:  (a) 
interests,  (b)  job  satisfaction,  (c)  military  values,  (d)  organizational  commitment,  and  (e) 
citizenship  motives.  We  developed  an  instrument  to  measure  the  following  proactive  cognitive 
processes:  (a)  self-efficacy,  (b)  expectancies,  and  (c)  goal  commitment.  On-line  cognitive 
processes  were  measured  by  developing  a  self-regulation  inventory  that  assessed  one’s  success  in 
self-regulating  when  performing  behaviors  relevant  to  each  performance  dimension.  Work  habits 
were  measured  by  developing  a  scale  that  assessed  the  extent  to  which  one’s  habitual  way  of 
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doing  things  facilitates  or  interferes  with  performance  on  each  dimension.  Knowledge  was 
measured  by  creating  a  situational  judgment  test  relevant  to  each  performance  dimension. 

Finally,  skill  was  measured  by  creating  a  past  behavior  record  that  asked  respondents  to  describe 
a  situation  in  which  they  exhibited  behavior  relevant  to  each  performance  dimension. 

In  the  following  sections,  we  describe  each  of  these  instruments  in  detail.  For  instruments 
developed  specifically  for  this  research,  we  describe  the  steps  that  were  taken  in  the  development 
procedure.  All  instruments  developed  for  this  research  were  based  on  input  from  and/or  reviewed 
by  cadets  and  midshipmen  in  the  Army,  Navy,  and  Air  Force  ROTC  programs  at  the  University 
of  Minnesota.  These  cadets  and  midshipmen  were  compensated  for  their  input  at  rates  of  $25  for 
a  one-hour  workshop,  $35  for  a  two-hour  workshop,  and  $50  for  a  three-hour  workshop. 

Performance  Rating  Instrument 

The  Performance  Rating  Instrument  was  developed  specifically  for  this  project  to  include 
items  relevant  to  each  of  the  targeted  performance  dimensions.  The  first  step  in  developing  this 
instrument  was  to  identify  the  performance  dimensions  to  be  measured.  We  conducted  a 
performance  domain  definition  workshop  with  three  cadets/midshipmen  from  each  ROTC 
branch  at  the  University  of  Minnesota  (a  total  of  nine  cadets),  in  which  we  determined  how  task, 
citizenship,  and  adaptive  performance  were  defined  for  ROTC  cadets  and  midshipmen.  We 
provided  participants  with  definitions  of  task,  citizenship,  and  adaptive  performance  dimensions 
from  Johnson’s  (2003)  performance  dimension  taxonomy  (see  Appendix  A).  We  then  asked 
them  to  identify  dimensions  that  are  particularly  important  and  salient  across  all  branches.  Our 
goal  was  to  identify  one  dimension  within  each  component  of  citizenship  performance  (i.e., 
personal  support,  organizational  support,  conscientious  initiative;  see  Borman,  Buck  et  al., 

2001),  plus  adaptive  performance  (i.e.,  dealing  with  uncertain  and  unpredictable  work  situations; 
see  Johnson,  2003;  Pulakos  et  al.,  2000).  We  also  asked  the  cadets  and  midshipmen  to  identify 
one  task  performance  dimension  from  the  taxonomy  that  was  relevant  and  had  the  same  meaning 
across  all  three  branches. 

The  workshop  participants  identified  decision  making/problem  solving  as  the  best  task 
performance  dimension  to  include  in  the  project.  The  participants  confirmed  that  adapting  to 
uncertain  and  changing  situations  is  relevant  to  the  role  of  ROTC  cadet/midshipman  and  is 
conceptually  distinct  from  decision  making/problem  solving,  so  it  was  determined  to  be  an 
appropriate  dimension  of  the  adaptive  performance  domain  for  measuring  in  this  project.  The 
participants  identified  maintaining  good  working  relationships  and  helping  others  as  potential 
dimensions  to  represent  personal  support,  organizational  commitment  to  represent  organizational 
support,  and  showing  initiative  to  represent  conscientious  initiative.  After  identifying  the  target 
performance  dimensions,  we  asked  the  participants  to  generate  lists  of  behaviors  that  may  be 
exhibited  by  ROTC  cadets  that  exemplify  high  levels  of  performance  on  those  dimensions. 

We  identified  a  set  of  items  for  the  draft  performance  rating  instrument  using  the 
behavioral  examples  collected  during  the  performance  domain  definition  workshop.  In 
developing  this  draft  set  of  items,  we  edited  and  included  items  from  other  performance  appraisal 
instruments  PDRI  has  created  that  measure  similar  dimensions  of  performance.  We  chose  the 
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best  subset  of  items  for  each  dimension  to  include  in  the  draft  instrument,  which  resulted  in  a 
draft  instrument  consisting  of  79  items  measuring  six  dimensions  and  overall  performance. 

We  then  reviewed  the  draft  performance  rating  instrument  in  a  separate  workshop 
consisting  of  nine  cadets  (three  from  each  ROTC  branch).  In  the  workshop,  we  explained  the 
purpose  of  the  research  and  asked  the  participants  to  review  the  performance  rating  form  and 
instructions.  We  asked  them  to  review  these  materials  for  understanding  and  appropriateness  of 
the  terminology,  ratability  of  items,  and  appropriateness  as  a  performance  evaluation  tool.  We 
also  asked  for  their  opinions  on  keeping  either  maintaining  good  working  relationships  or 
helping  others  as  the  dimension  for  personal  support.  The  participants  thought  helping  others  was 
less  observable  because  it  is  often  a  one-on-one  behavior  and  they  saw  it  as  a  subset  of 
maintaining  good  working  relationships.  Thus,  we  kept  maintaining  good  working  relationships 
as  the  performance  dimension  representing  personal  support  and  dropped  helping  others. 

The  materials  were  revised  based  on  the  information  gathered  during  these  workshops. 
The  final  performance  rating  instrument  consisted  of  40  behavioral  statements  measuring  the  five 
performance  dimensions  and  overall  performance.  The  instructions  asked  raters  to  rate  the 
individual  on  the  extent  to  which  each  behavioral  statement  describes  him  or  her  on  a  scale 
ranging  from  1  =  not  at  all  to  5  =  to  a  very  great  extent,  with  a  not  observed  option.  Rater 
training  was  provided  in  written  form,  focusing  on  how  to  use  the  rating  scales  and  awareness  of 
common  rating  errors.  The  instrument  was  designed  to  be  completed  online.  The  performance 
rating  instrument  is  included  in  Appendix  B. 

Cognitive  Ability  Measure 

General  cognitive  ability  was  measured  with  the  Wonderlic  Personnel  Test  (WPT) 
(Wonderlic  Personnel  Test,  1992).  The  WPT  is  a  12-minute  test  that  assesses  vocabulary, 
arithmetic  reasoning,  and  spatial  relations.  It  consists  of  50  multiple-choice  questions  of 
increasing  difficulty  and  has  been  shown  to  be  both  a  reliable  and  construct- valid  measure  of  g 
(Murphy,  1 984).  Normative  data  for  the  WPT  indicate  a  mean  score  of  2 1 .06  (SD  =  7.12)  and 
test-retest  reliability  ranging  from  .82  to  .94. 

Personality  Measure 

The  Johnson  (2003)  performance  prediction  model  was  used  to  generate  hypotheses 
about  the  personality  variables  that  were  expected  to  influence  the  direct  determinants  of  each 
performance  dimension.  Scales  from  the  International  Personality  Item  Pool  (IPIP)  (2001; 
http://ipip.ori.org),  described  in  Goldberg  (1999),  were  chosen  to  cover  the  range  of  personality 
scales  that  were  needed.  Use  of  these  items  is  free  to  investigators  for  use  in  their  research,  and 
the  IPIP  item  pool  includes  scales  from  virtually  all  prominent  personality  inventories.  Lim  and 
Ployhart  (2006)  provided  construct  validity  support  for  the  IPIP.  Seventeen  scales  were  selected 
for  inclusion  in  the  project,  some  of  which  were  facets  of  the  Big  Five  and  others  of  which  were 
simply  specific  personality  constructs  that  were  expected  to  be  related  to  one  or  more 
performance  dimensions.  The  number  of  items  in  each  scale  ranged  from  8  to  14.  Participants 
were  instructed  to  rate  the  extent  to  which  each  item  described  them  with  a  rating  scale  ranging 
from  1  =  Definitely  True  to  5  =  Definitely  False. 
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Motivational  Trait  Questionnaire 

The  Motivational  Trait  Questionnaire  (MTQ)  was  developed  to  measure  individual 
differences  in  traits  relevant  to  work  motivation  (Heggestad  &  Kanfer,  2000).  The  MTQ 
measures  four  broad  traits:  personal  mastery  and  competitive  excellence  are  facets  of 
achievement  motivation,  and  achievement  anxiety  and  failure  avoidance  are  facets  of  anxiety. 
This  82-item  instrument  uses  a  6-point  response  scale  ranging  from  1  =  Very  UNTRUE  of  Me  to 
6=  Very  TRUE  of  Me. 

Social  Insight 

We  included  the  Social  Insight  scale  from  the  PDRI  Social  Competence  Inventory, 

Version  2  (SCI-2)  (Schneider,  2001).  The  SCI-2  is  a  self-report  measure  of  the  broad  social 
competence  domain.  Its  items  are  rated  on  a  Likert-type  scale,  ranging  from  1  =  Definitely  False 
to  5  =  Definitely  True.  This  scale  measures  the  extent  to  which  a  person  believes  he  or  she  is  able 
to  (a)  discern  the  motivations,  feelings,  and  intentions  underlying  people’s  behavior  by  correctly 
interpreting  behavioral  cues;  (b)  see  things  from  others’  perspectives;  and  (c)  accurately  predict 
others’  behavior  (Schneider  &  Johnson,  2005).  This  scale  was  found  to  be  distinct  from  the 
interpersonal  personality  trait  domain  and  was  a  good  predictor  of  several  interpersonal 
performance  dimensions  in  Schneider  and  Johnson  (2005),  so  we  expected  that  it  would  be  a 
good  predictor  of  maintaining  good  working  relationships.  The  23-item  scale  was  reduced  to 
nine  items  by  retaining  the  items  that  had  the  largest  correlations  with  Schneider  and  Johnson’s 
performance  dimensions. 

Experience  Inventory 

An  experience  inventory  was  created  in  which  respondents  were  presented  with  the  items 
from  the  performance  rating  instrument  and  asked  to  rate  the  frequency  with  which  they  have 
engaged  in  each  behavior  in  the  last  few  years.  The  response  scale  was  taken  from  Pulakos, 
Schmitt,  Dorsey,  Arad,  Hedge,  and  Borman  (2002),  with  1  =  Never,  2  =  Once  or  Twice,  3  = 
Several  Times,  and  4  =  Frequently  or  Routinely.  The  Experience  Inventory  is  included  in 
Appendix  C. 

Action  Control  Scale 

Kuhl’s  (1985)  action  control  theory  focuses  on  the  translation  of  an  intention  to  an  action 
through  self-regulatory  processes.  According  to  action  control  theory,  self-regulatory  skill  is 
partially  determined  by  an  individual’s  action-state  orientation.  Action-state  orientation  is 
conceptualized  by  Kuhl  as  a  continuous  individual-differences  variable.  More  action-oriented 
individuals  are  better  able  to  devote  their  attention  to  the  current  goal.  More  state-oriented 
individuals  tend  to  ruminate  on  alternative  goals  or  emotional  states,  reducing  the  cognitive 
resources  available  for  striving  for  the  current  goal.  Diefendorff,  Hall,  Lord,  and  Strean  (2000) 
evaluated  the  construct  validity  of  a  revised  version  of  a  measure  of  action-state  orientation,  the 
Action  Control  Scale  (Kuhl,  1 994).  This  scale  measures  three  dimensions  of  action-state 
orientation:  (a)  preoccupation  (degree  to  which  individuals  detach  from  thoughts  about 
interfering  goals),  (b)  hesitation  (difficulty  in  initiating  goal-directed  action),  and  (c)  volatility 
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(degree  to  which  individuals  become  distracted  when  working  on  a  task).  Diefendorff  et  al. 
conducted  confirmatory  factor  analyses  to  confirm  the  three- factor  model  and  used  the  results  to 
create  a  revised  version  of  the  Action  Control  Scale.  The  revised  version  consists  of  22  items 
measuring  the  three  scales.  They  found  that  the  action  control  scales  contributed  significant 
variance  beyond  a  measure  of  the  Big  Five  personality  dimensions  to  the  prediction  of  supervisor 
ratings  of  task  performance  and  several  organizational  citizenship  performance  dimensions.  We 
created  an  instrument  for  this  research  by  writing  instructions  and  assembling  the  items  from 
Diefendorff  et  al.  into  a  questionnaire  format.  The  revised  Action  Control  Scale  is  included  in 
Appendix  D. 

Military  Motives  Scale 

The  Military  Motives  Scale  (MMS)  measured  values,  interests,  and  attitudes  that  were 
expected  to  predict  proactive  cognitions  relevant  to  the  target  behaviors.  The  MMS  measured  job 
satisfaction,  military  values,  organizational  commitment,  and  interests. 

The  Job  Satisfaction  scale  was  constructed  by  taking  13  job  satisfaction  items  from  the 
Sample  Survey  of  Military  Personnel  (Schumm,  Gade,  &  Bell,  2003a)  that  appeared  to  be 
relevant  to  the  role  of  ROTC  cadet/midshipman.  During  a  review  workshop  consisting  of  eight 
cadets  and  midshipmen,  these  items  were  revised  and  two  additional  items  were  added  to  create  a 
1 5-item  scale  for  the  final  instrument.  Respondents  were  asked  how  satisfied  they  are  with  the 
aspect  of  their  role  as  an  ROTC  cadet/midshipmen  represented  by  the  item,  with  a  response  scale 
ranging  from  1  =  very  dissatisfied  to  5  =  very  satisfied. 

The  Military  Values  Scale  was  constructed  by  taking  15  professional  values  items  from 
the  Sample  Survey  of  Military  Personnel  (Schumm,  Gade,  &  Bell,  2003b)  that  appeared  to  be 
relevant  to  the  role  of  ROTC  cadet/midshipman.  Some  of  the  items  were  revised  during  the 
review  workshop  to  make  them  more  applicable  to  ROTC.  Respondents  were  asked  how 
important  each  value  is  to  them,  with  a  response  scale  ranging  from  0  =  not  at  all  important  to  4 
=  very  important. 

The  Organizational  Commitment  scale  was  constructed  by  taking  four  items  measuring 
affective  commitment  and  four  items  measuring  continuance  commitment  identified  by  Gade, 
Tiggle,  and  Schumm  (2003).  One  item  was  revised  for  clarity  during  the  workshop.  Respondents 
were  asked  to  indicate  the  extent  to  which  they  agreed  with  each  statement,  with  a  response  scale 
ranging  from  1  =  strongly  disagree  to  5  =  strongly  agree. 

An  interest  inventory  was  created  measuring  interest  in  taking  part  in  situations  that 
demand  behavior  relevant  to  the  target  performance  dimensions.  We  adapted  the  behaviors  on 
the  performance  rating  instrument,  consolidating  behaviors  where  it  made  sense,  to  create  a  set 
of  33  behavioral  statements.  For  each  behavior,  respondents  were  asked  to  imagine  they  are  in  a 
situation  where  they  have  to  engage  in  the  behavior  and  rate  the  extent  to  which  they  would 
enjoy  that  situation.  The  response  scale  ranged  from  1  =  I  would  dislike  this  situation  very  much 
to  5  =  I  would  like  this  situation  very  much.  This  scale  was  created  after  considerable  input  from 
cadets  and  midshipmen  participating  in  the  review  workshop.  The  MMS  is  included  in  Appendix 
E. 
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Citizenship  Motives  Scale 

Rioux  and  Penner  (2001)  created  a  Citizenship  Motives  Scale  (CMS)  to  measure  three 
different  motives  for  engaging  in  organizational  citizenship  behaviors:  (a)  prosocial  values,  (b) 
organizational  concern,  and  (c)  impression  management.  The  prosocial  values  scale  consists  of 
10  items  measuring  a  need  to  be  helpful  and  a  desire  to  build  positive  relationships.  The 
organizational  concern  scale  consists  of  1 0  items  measuring  a  desire  for  the  organization  to  do 
well  and  a  desire  to  show  pride  in  and  commitment  to  the  organization.  The  impression 
management  scale  consists  of  1 0  items  measuring  a  desire  to  avoid  looking  bad  to  others  and  to 
obtain  rewards. 

We  adapted  this  scale  to  our  own  research  by  presenting  participants  with  a  definition  and 
example  behaviors  for  each  of  the  three  citizenship  performance  dimensions  we  targeted.  For 
each  performance  dimension,  we  presented  the  30  CMS  items,  asking  how  important  each 
motive  statement  is  in  the  decision  to  engage  in  behaviors  that  represent  performance  on  that 
dimension.  The  response  scale  ranged  from  1  =  not  at  all  important  to  5  =  very  important.  The 
CMS  is  included  in  Appendix  F. 

Proactive  Cognitions  Scale 

The  Proactive  Cognitions  Scale  is  a  1 1 1  -item  scale  developed  for  this  research.  It 
includes  questions  about  expectations,  self-efficacy,  and  goal  commitment.  We  measured 
expectancy  by  asking  respondents  to  indicate  the  probability  of  attaining  certain  outcomes  by 
engaging  in  a  specific  behavior.  Specifically,  the  respondent  is  presented  with  each  of  the 
behaviors  from  the  performance  rating  instrument.  For  each  behavior,  the  respondent  is  asked  to 
consider  the  probability  of  achieving  one  or  more  of  the  following  outcomes  if  the  behavior  is 
performed: 


The  unit  would  be  more  successful. 

I  would  receive  personal  recognition  (e.g.,  praise,  award). 

My  unit  would  be  favorably  recognized. 

I  would  advance  in  my  career. 

My  peers  would  be  pleased. 

I  would  avoid  punishment. 

I  would  benefit  personally  at  a  later  time  (e.g.,  a  future  favor,  time  off). 

Next,  respondents  are  asked  to  rate  the  probability  of  achieving  one  or  more  of  the  above 
outcomes  using  a  scale  ranging  from  1  =  highly  improbable  to  5  =  highly  probable. 

We  measured  behavior-specific  self-efficacy  by  presenting  the  same  set  of  items,  with 
respondents  indicating  how  effectively  they  would  exhibit  each  behavior  using  a  scale  ranging 
from  1  =  highly  ineffective  to  5  =  highly  effective.  Pulakos  et  al.  (2002)  created  a  self-efficacy 
scale  for  adaptive  performance  using  this  procedure. 
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We  measured  goal  commitment  by  presenting  the  same  set  of  items  and  asking 
respondents  to  indicate  how  committed  they  are  to  performing  the  behavior  during  the  following 
semester.  They  used  the  following  scale  to  indicate  their  level  of  commitment: 

1  =  Not  at  All  Committed  - 1  definitely  will  not  do  this. 

2  =  Slightly  Committed  - 1  probably  will  not  do  this. 

3  =  Somewhat  Committed  - 1  may  or  may  not  do  this. 

4  =  Committed  - 1  will  probably  do  this. 

5  =  Very  Committed  - 1  will  definitely  do  this. 

The  Proactive  Cognitions  Scale  is  included  in  Appendix  G. 

Self-Regulation  Inventory 

Research  on  self-regulation  is  diverse,  spanning  several  fields  of  psychological  study 
(Como,  2004).  It  includes  the  study  of  emotion  control  as  part  of  social  intelligence; 
investigations  of  interventions  to  improve  physical  health;  the  examination  of  volitional 
strategies  in  both  educational  and  work  settings;  and  research  on  goals,  self-efficacy,  feedback, 
and  goal-performance  discrepancies  (and  their  reduction).  Given  this  broad  and  scattered  work,  it 
is  not  surprising  that  there  is  little  agreement  on  how  to  define  self-regulation  (Vancouver  & 

Day,  2005)  and  many  models  of  self-regulation  have  been  developed  (Kozlowski  &  Bell,  2006). 
Although  many  of  these  models  include  three  interdependent  processes — self-monitoring,  self- 
evaluation,  and  self-reactions  (Donovan,  2002;  Kanfer  &  Heggestad,  1997;  Kozlowski  &  Bell, 
2006) — numerous  constructs  have  been  proposed  to  be  components  of  these  processes  and  other 
aspects  of  self-regulation  (Karoly,  Boekaerts,  &  Maes,  2005).  Within  industrial  and 
organizational  psychology,  .  .relatively  little  attention  has  been  paid  to  assessing  whether  the 
constructs  represent  meaningful  mechanisms  involved  in  goal  processes  and/or  are 
unconfounded  with  other  key  constructs...  The  cumulative  result  of  these  issues  is  little  progress 
in  terms  of  understanding  self-regulation’s  role  in  work  processes  or  rigorous  assessments  of 
interventions  based  on  a  self-regulation  perspective.”  (Vancouver  &  Day,  2005,  p.  157).  The 
relationships  among  self-regulatory  constructs  are  not  well  known  (Sperling,  Howard,  Staley,  & 
DuBois,  2004).  Thus,  the  selection  of  measures  to  assess  self-regulation  is  not  straightforward. 

Theory  and  research  has  suggested  that  individual  differences  in  self-regulation  can  be 
conceptualized  both  in  terms  of  traits  and  as  responses  to  specific  situations  (Kanfer  & 

Heggestad,  1997,  1999;  Sarason,  Sarason,  Keefe,  Hayes,  &  Shearin,  1986).  Measures  of  both  are 
included  in  the  present  research.  The  MTQ  described  earlier  represents  the  trait  approach  to  self¬ 
regulation  and  the  Self-Regulation  Inventory  represents  the  situational  response  approach.  In  the 
following  section,  we  explicate  the  theoretical  rationale  for  the  items  included  on  the  Self- 
Regulation  Inventory. 

Several  approaches  have  been  used  when  examining  self-regulation  as  a  response  to  a 
specific  situation.  These  include  observing  self-regulatory  activities,  assessing  the  mechanics  of 
self-evaluation,  measuring  motivational  or  volitional  skills,  quantifying  affective  (e.g.,  emotional 
states)  or  cognitive  self-reactions  (e.g.,  self-efficacy),  and  evaluating  the  direction  of  attention. 
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Observation  is  one  approach  that  has  been  used  to  assess  self-regulatory  activities 
(Kozlowski  &  Bell,  2006).  For  example,  time  spent  reviewing  feedback  has  been  used  to 
measure  self-evaluation  (Kozlowski  &  Bell,  2006).  An  obvious  advantage  of  this  approach  is 
that  it  does  not  rely  on  individuals’  recognition  of  their  own  self-regulatory  processes.  However, 
this  methodology  requires  the  assumption  that  the  time  spent  looking  at  certain  materials 
accurately  reflects  use  of  self-regulatory  mechanisms  and  it  can  only  be  used  in  contexts  where 
researchers  have  control  over  the  materials  available  for  individuals  to  use  while  working.  In  the 
present  research,  self-reports  of  self-regulatory  activities  will  therefore  be  used.  Research  in 
educational  settings  has  found  associations  between  such  self-reports  and  academic  performance. 
For  example,  self-reports  of  self-monitoring  were  correlated  with  course  grades  and  subjective 
measures  of  academic  achievement  (Perry,  Hladkyj,  Pekrun  &  Pelletier,  2001) 

Researchers  seeking  to  understand  the  effects  of  goal  setting  on  self-regulation  and 
performance  have  focused  on  examining  the  mechanics  of  self-evaluation.  Prominent  in  this 
work  are  the  constructs  of  goals,  performance,  goal-performance  discrepancies,  and  goal 
revision.  Each  variable  considered  is  quantified  independently.  For  example,  the  content  and/or 
level  of  the  goal  and  the  level  of  performance  are  assessed.  Repeated  measures  designs  have 
been  used  to  evaluate  the  dynamic  interplay  of  goals  and  performance.  This  research  has  yielded 
insight  into  how  individuals  manage  their  goal-directed  behavior  over  time,  but  it  is  best  suited 
for  the  study  of  well-defined  and  limited  tasks,  such  as  those  used  in  laboratory  studies  or  in 
training  courses.  In  a  complex  context  such  as  the  one  examined  in  the  present  research, 
individuals  pursue  diverse  and  multiple  tasks,  making  it  problematic  to  assess  each  component 
utilized  in  self-evaluation. 

Other  scholars  have  focused  on  understanding  volitional  strategies  or  motivational  skills. 
For  example,  Kuhl  (1985;  Kuhl  &  Fuhrmann,  1998)  has  assessed  a  variety  of  volitional 
strategies.  Kanfer  and  Heggestad  (1997, 1999)  have  argued  that  malleable  and  domain-specific 
motivational  skills  play  an  important  role  in  the  goal-striving  process  (Kanfer  &  Heggestad, 
1997).  “Motivational  skills  pertain  to  the  self-regulatory  strategies  used  by  individuals  during 
goal  striving.  Self-regulatory  strategies  are  defined  as  integrated  patterns  of  response  to 
difficulties  or  anticipated  difficulties  in  goal-directed  action.  Such  skills  provide  agentic  control 
of  affect,  cognition,  and  behavior  that  facilitate  goal  accomplishment.”  (Kanfer  &  Heggestad, 
1997,  p.  39). 

While  acknowledging  that  other  motivational  skills  have  been  proposed,  Kanfer  and 
Heggestad  (1997)  focused  on  two:  (a)  emotion  control,  and  (b)  motivation  control.  They  note 
that  little  progress  has  been  made  in  measuring  these  skills.  Challenges  include  the  need  to  assess 
“on-line”  self-regulatory  processes  using  diary  procedures  (Maes  &  Karoly,  2005)  and  the 
likelihood  that  accurate  measurement  requires  assessing  individuals  when  they  are  experiencing 
difficulty  in  goal-striving.  Motivational  skills  are  most  likely  to  be  used,  and  therefore  amenable 
to  assessment,  under  specific  conditions.  Emotion  control  is  most  important  when  tasks  are  new 
and  negative  emotions,  such  as  concerns  about  failure,  self-dissatisfaction,  and  worry  about 
performance,  must  be  managed.  Unusually  stressful  or  boring  tasks  are  thought  to  be  times  when 
training  to  strengthen  motivation  control  might  be  helpful  (Kanfer  &  Heggestad,  1997).  Lack  of 
experimental  control  creates  considerable  uncertainty  about  when  these  conditions  will  arise  in  a 
field  setting,  making  it  difficult  to  accurately  assess  motivational  skills.  Failure  to  observe 
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emotion  control  or  motivation  control  may  be  an  indication  that  self-regulatory  challenges 
requiring  the  use  of  volitional  strategies  have  not  arisen,  rather  than  a  sign  that  a  person  lacks 
motivational  skills. 

In  order  to  take  into  account  the  domain-specific  nature  of  motivational  skills  and  their 
“on-line”  use,  we  posed  self-regulatory  questions  in  the  context  of  specific  dimensions  of  job 
performance.  Individuals  were  asked  to  think  about  times  in  the  past  month  they  had  engaged  in 
behaviors  related  to  specific  dimensions  of  performance.  For  each  dimension  of  performance,  a 
definition  was  provided.  Subjects  were  then  asked  to  respond  to  17  items  assessing  four  aspects 
of  self-regulatory  activity:  (a)  negative  affect,  (b)  mental  focus,  (c)  the  impact  of  self-regulatory 
failure,  and  (d)  work  habits.  For  each  item,  ratings  were  made  on  a  5-point  Likert-type  agreement 
scale  ranging  from  1=  Strongly  Disagree  to  5  =  Strongly  Agree.  These  aspects  of  self-regulatory 
activity  are  described  in  the  following  paragraphs. 

Several  researchers  have  attempted  to  overcome  the  problem  of  determining  whether  a 
failure  to  observe  emotion  control  is  an  indication  that  the  situation  did  not  require  the  use  of 
volitional  strategies  or  a  sign  of  poor  motivational  skills  by  measuring  affect,  rather  than  emotion 
control  (Diefendorff,  Richard,  &  Robie,  2005;  Kanfer,  Ackerman,  Murtha,  Dugdale,  &  Nelson, 

1 994;  Kozlowski  &  Bell,  2006).  Self-reactions  are  one  of  the  three  broadly  recognized  self- 
regulatory  constructs.  They  include  both  affective  reactions  (i.e.,  self-satisfaction  or 
dissatisfaction)  and  cognitive  task-related  self-perceptions  (i.e.,  self-efficacy).  Frequent 
experiences  of  negative  affect  while  working  on  a  task  are  a  signal  that  individuals  are  aware  of 
falling  short  of  their  desired  behaviors  and  have  not  successfully  utilized  volitional  skills  or 
strategies  to  address  their  potentially  destructive  associated  negative  dissatisfactions.  Previous 
research  has  found  that  negative  affect  is  positively  associated  with  having  difficult  goals 
(Kanfer  et  al.,  1994)  and  negatively  related  to  basic  knowledge  acquisition  and  basic 
performance  following  training  (Kozlowski  &  Bell,  2006).  Note  that  positive  affect  may  be 
experienced  either  because  no  self-regulatory  challenges  are  encountered  or  because  volitional 
strategies  have  been  deployed  successfully.  Thus,  the  interpretation  of  experiences  of  positive 
affect  while  working  is  ambiguous  and  was  not  measured  in  this  research.  Four  items  assessing 
experiences  of  negative  affect  were  included  in  the  measure  of  self-regulatory  activity  included 
in  this  research.  These  items  were  adapted  from  instruments  developed  by  Kanfer  et  al.  (1994) 
and  Diefendorff  et  al.  (2005). 

A  number  of  researchers  have  investigated  intrusive  thoughts  individuals  report  having 
while  working  on  tasks  (Diefendorff  et  al.,  2005;  Kanfer  et  al.,  1994;  Kozlowski  &  Bell,  2006; 
Sarason  et  al.,  1986).  Intrusive  thoughts  are  ones  that  do  not  directly  pertain  to  doing  the  task. 
They  may  be  completely  unrelated  to  the  task,  or  may  focus  on  aspects  of  the  task  that  are 
unrelated  to  the  task.  For  example,  off-task  thoughts  may  include  contemplating  other  activities. 
Intrusive  on-task  thoughts  include  worrying  about  performance  and  task  difficulty.  Depending  on 
their  content,  intrusive  thoughts  could  be  indicative  of  a  variety  of  self-regulatory  activities, 
including  a  failure  of  attention  control,  attempts  to  identify  methods  to  address  goal-performance 
discrepancies  identified  through  self-evaluation,  or  a  general  sign  that  goal  striving  has  ceased. 
The  measure  used  in  the  present  research  attempts  to  assess  the  latter.  Past  research  has  been 
mixed  on  the  relationship  between  intrusive  thoughts  and  performance.  In  one  study,  intrusive 
thoughts  about  the  task  were  negatively  related  to  task  performance,  but  off-task  or  wandering 
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thoughts  were  not  (Sarason  et  al.,  1986).  Mental  focus  was  related  to  course  performance  in  one 
study  (Lee,  Sheldon,  &  Turban,  2003),  but  not  a  second  (Diefendorff  et  al.,  2005).  Four  of  the  1 7 
self-regulatory  activity  items  in  our  instrument  assessed  mental  focus.  Ideas  for  item  content 
were  based  on  items  included  in  measures  used  by  Kanfer  et  al.  (1994)  and  Diefendorff  et  al. 
(2005)  in  studies  of  training  performance  on  computer  tasks  and  course  performance, 
respectively. 

Six  items  were  developed  to  measure  the  impact  of  self-regulatory  failure  on 
performance.  These  items  attempt  to  more  directly  evaluate  poor  use  of  self-regulatory  skills, 
rather  than  to  infer  that  individuals  were  experiencing  self-regulatory  challenges  from 
assessments  of  affect  and  cognition.  The  items  assessed  respondents’  agreement  that  poor 
emotion,  motivation,  or  behavioral  control  hurt  their  performance. 

The  three  final  items  used  to  assess  self-regulatory  success  focused  on  the  extent  to  which 
the  respondents’  work  habits  helped  or  hindered  performance.  To  the  extent  work  habits  fostered 
effective  performance,  self-regulatory  processes  during  goal  striving  would  be  automatic, 
reducing  the  need  to  use  controlled,  self-regulatory  activities.  Reports  that  work  habits  were  not 
conducive  to  good  performance  are  a  sign  that  self-regulatory  activities  were  needed. 

The  Self-Regulation  Inventory  was  administered  online  approximately  once  a  month  for 
four  months.  Respondents  were  instructed  to  think  back  over  the  past  month  and  recall  the  most 
recent  ROTC-related  situation  when  they  performed  a  behavior  relevant  to  each  performance 
dimension.  They  then  were  instructed  to  keep  that  instance  in  mind  when  responding  to  the 
questions.  For  example,  respondents  were  presented  with  the  definition  of  decision 
making/problem  solving  and  several  sample  behaviors  (i.e.,  the  behaviors  from  the  performance 
rating  instrument).  Then  they  were  asked  to  think  about  when  they  most  recently  experienced  a 
situation  in  which  they  could  make  a  decision  or  solve  a  problem  related  to  ROTC.  They  were 
asked  to  take  a  moment  to  think  about  that  time,  visualize  and  walk  themselves  through  the 
experience,  and  recall  what  the  situation  was  leading  up  to  the  experience  and  what  they  did. 

Then  they  rated  their  agreement  with  each  of  the  statements  as  they  related  to  that  experience. 

The  Self-Regulation  Inventory  is  included  in  Appendix  H. 

Work  Habits  Scale 

“Work  habits  are  patterns  of  behavior  that  people  learn  over  time  and  that  can  either 
facilitate  or  interfere  with  the  performance  of  behaviors  that  contribute  to  the  accomplishment  of 
organizational  goals”  (Motowidlo  et  al.,  1997,  p.  79).  They  reflect  an  individual’s  characteristic 
way  of  handling  a  class  of  situations  that  has  developed  over  time  through  the  pairing  of  a 
sequence  of  actions  that  constitute  a  satisfactory  response  to  the  cue  of  a  situation  (Verplanken  & 
Orbell,  2003).  Work  habits  can  be  viewed  as  a  type  of  scripted  behavior  where  the  script  is  the 
cognitive  structure  capturing  the  learned  associations  between  behaviors  and  goals  (Verplanken 
&  Orbell,  2003).  Individuals  are  likely  to  use  work  habits  automatically,  unconsciously,  and  in 
an  uncontrolled  fashion,  (Verplanken  &  Orbell,  2003).  Although  work  habits  are  likely  to  be 
characterized  by  efficiency  (Verplanken  &  Orbell,  2003),  they  may  not  represent  the  best  or  most 
effective  ways  of  handling  situations  at  work  (Motowidlo  et  al,  1997). 
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The  development  of  measures  of  habits  has  been  problematic.  Most  researchers  have 
focused  on  assessing  the  frequency  with  which  behaviors  have  been  performed  in  the  past,  but 
this  approach  focuses  on  the  repetitive  nature  of  habits  without  accounting  for  their  other  key 
attributes  (Verplanken,  2004).  Other  assessment  approaches  that  have  been  developed  are 
difficult  to  utilize  in  work  settings.  For  example,  response  frequency  methods  can  only  be  used 
to  evaluate  behaviors  implemented  in  multiple  situations,  must  be  administered  in  a  controlled 
setting,  and  require  extensive  pilot  work  for  each  behavior  assessed  (Verplanken  &  Orbell, 

2003).  Some  self-report  measures  of  habit  strength  have  been  criticized  for  relying  on  single  item 
measures  (Verplanken  &  Orbell,  2003).  Longer  and  more  reliable  alternatives,  such  as  the  Self- 
Report  Habit  Index  (Verplanken  &  Orbell,  2003),  assess  a  limited  number  of  quite  specific 
behaviors  (e.g.,  bus  use),  making  them  unsuitable  to  the  measurement  of  the  diverse  and 
numerous  habits  individuals  might  use  in  work  situations.  Therefore,  a  new  kind  of  work  habits 
measure  was  developed  for  the  present  research. 

The  Work  Habits  Scale  does  not  attempt  to  directly  measure  individual’s  work  habits. 
Instead,  it  measures  individuals’  perceptions  of  the  extent  to  which  their  habitual  way  of  doing 
things  helps  or  hinders  their  performance  on  the  37  behaviors  that  describe  the  five  performance 
dimensions  we  assessed.  Instructions  for  the  instrument  provide  a  definition  of  work  habits  and 
examples  illustrating  how  work  habits  may  facilitate  or  interfere  with  performance.  Respondents 
then  are  given  the  definition  of  one  of  the  performance  dimensions  and  prompted  to  think  first 
about  the  habits  or  routines  that  make  performing  this  dimension  easier  and  next  to  reflect  on  the 
habits  or  routines  that  interfere  with  performing  this  dimension  effectively.  Finally,  respondents 
are  asked  to  rate  behaviors  indicative  of  that  performance  dimension  on  a  7-point  scale  reflecting 
the  extent  to  which  their  habitual  way  of  doing  things  helps  or  interferes  with  performing  the 
behavior  (1  =  Interferes  a  great  deal;  1  =  Helps  a  great  deal).  The  process  of  presenting 
definitions  of  performance,  reflecting  on  work  habits,  and  rating  behaviors  is  repeated  for  the 
remaining  four  scales.  The  Work  Habits  Scale  is  included  in  Appendix  I. 

Situational  Judgment  Test 

Knowledge  relevant  to  the  performance  dimensions  chosen  for  evaluation  was  assessed 
with  a  Situational  Judgment  Test  (SJT)  that  presents  the  individual  with  a  situation  and  a  set  of 
response  options  varying  in  their  effectiveness.  SJTs  are  a  type  of  job  knowledge  test  (Legree, 
Psotka,  Tremble,  &  Bourne,  2005;  McDaniel  &  Nguyen,  2001;  Salgado,  Viswesvaran,  &  Ones, 
2001 ;  Schmidt  &  Hunter,  1993),  and  should  be  effective  in  measuring  knowledge  relevant  to  the 
performance  dimensions  in  this  research.  A  challenge  in  using  SJTs  to  measure  knowledge 
relevant  to  specific  constructs  is  that  SJTs  are  typically  characterized  by  low  internal  consistency 
and  multidimensionality,  even  when  attempts  are  made  to  measure  a  specific  construct 
(Gillespie,  Oswald,  Schmitt,  Manheim,  and  Kim,  2002).  Attempts  have  been  made  to  create  SJTs 
that  measure  specific  constructs,  but  these  constructs  have  typically  been  individual  difference 
variables  such  as  personality  traits  (e.g.,  Motowidlo,  Diesch,  &  Jackson,  2003;  Ployhart  &  Ryan, 
2000;  Trippe  &  Foti,  2003).  SJTs  are  better  thought  of  as  knowledge  measures  (Chan  &  Schmitt, 
2005;  Kim,  Schmitt,  Oswald,  Gillespie,  &  Ramsay,  2003),  so  they  should  be  targeted  toward  the 
constructs  they  are  intended  to  predict.  Rather  than  targeting  the  SJT  toward  a  multidimensional 
criterion  of  job  performance,  however,  it  should  be  targeted  toward  specific  dimensions  of 
performance.  In  this  project,  a  construct-oriented  test  construction  approach  was  used  to  develop 
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an  SJT  designed  to  measure  the  five  target  performance  dimensions.  We  took  the  following  steps 
to  create  the  SJT : 

Step  1:  Situation  generation.  First,  two  situation  generation  workshops  were  conducted, 
involving  12  cadets  and  midshipmen  representing  each  ROTC  branch.  In  each  workshop,  cadets 
were  presented  with  definitions  of  the  target  dimensions.  They  were  asked  to  generate  situational 
item  stems  by  writing  brief  descriptions  of  situations  they  had  witnessed  or  experienced,  in 
which  a  cadet/midshipman  demonstrated  a  high  or  low  level  of  performance  on  one  of  the  target 
dimensions.  These  descriptions  were  later  edited  for  grammar  and  clarity.  In  addition,  situations 
similar  in  content  were  combined,  retaining  the  core  elements  of  the  situation  and  the  knowledge 
involved  in  handling  the  situation. 

Step  2:  Response  generation.  Next,  two  response  generation  workshops  were  conducted, 
involving  12  cadets  and  midshipmen.  These  workshops  included  first-  and  second-year  cadets  as 
well  as  more  experienced  cadets  to  better  generate  response  options  that  would  appear  realistic 
but  vary  in  effectiveness.  During  the  workshops,  participants  were  asked  to  perform  two  tasks. 
First,  they  reviewed  the  situations  generated  in  the  preceding  workshops  to  ensure  that  the 
situations  were  clear,  realistic,  relevant,  non-obvious,  and  called  for  a  response  that  represented 
performance  on  the  target  dimension.  If  a  situation  did  not  meet  these  criteria,  participants  were 
asked  to  suggest  ways  to  improve  the  situation  description.  Second,  participants  were  asked  to 
write  a  one-  or  two-sentence  response  describing  how  they  would  handle  each  of  the  situations. 
Participants  were  instructed  to  only  write  responses  that  included  behavior  relevant  to  the 
targeted  performance  dimension.  After  the  response  generation  workshops,  we  edited  the 
response  options  for  grammar,  redundant  content,  construct  relevance,  and  clarity.  The  outcome 
of  this  work  was  96  realistic  situations  targeted  toward  the  performance  dimensions  we 
identified,  and  a  representative  sampling  of  the  kinds  of  responses  cadets  might  make  in  these 
situations,  spanning  the  continuum  from  very  effective  to  ineffective  actions. 

Step  3:  Response  options  selection.  To  choose  the  best  response  options  for  each 
situation,  the  responses  were  scaled  according  to  their  effectiveness  level.  Effectiveness  ratings 
were  collected  from  1 5  third-  and  fourth-year  cadets  in  two  workshops.  Each  participant  was 
instructed  to  read  the  situational  item  stem  and  all  of  the  possible  response  options.  Then  the 
participants  rated  the  effectiveness  of  each  response  option  using  a  7-point  scale  ranging  from  1 
=  very  ineffective  to  7  =  very  effective.  We  calculated  the  mean  and  standard  deviation  of  the 
effectiveness  ratings  assigned  to  each  response  option  and  used  these  data  to  choose  the  items  to 
include  in  the  SJT  and  the  response  options  to  be  included  for  each  item.  We  retained  items  for 
which  four  response  options  were  available  that  (a)  had  mean  effectiveness  ratings  that 
represented  a  broad  range  of  effectiveness  levels  and  (b)  showed  good  agreement  among  raters  in 
the  level  of  effectiveness  represented  by  each  response  option  (i.e.,  rwg  >  .50).  We  eliminated  28 
items  based  on  these  criteria.  This  left  68  items,  each  with  four  response  options. 

To  increase  our  pool  of  potential  items,  we  included  seven  items  written  to  measure 
“dealing  effectively  with  unpredictable  or  changing  work  situations”  in  an  SJT  created  by 
Pulakos  and  Dorsey  (2000).  This  increased  the  pool  of  potential  items  to  75. 
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Step  4:  Relevance  ratings.  To  evaluate  the  construct  relevance  of  each  item,  1 0 
experienced  10  psychologists  (mean  years  of  experience  working  as  an  10  psychologist  =  16.84, 
SD  =  9.71;  8  with  Ph.D.,  2  with  M.A.)  completed  a  survey  in  which  they  rated  the  extent  to 
which  each  situation  would  elicit  a  response  that  was  relevant  to  each  of  the  five  performance 
dimensions  and  the  extent  to  which  each  response  option  was  relevant  to  the  targeted  dimension. 
Items  were  chosen  for  inclusion  in  the  final  SJT  if  they  met  the  following  criteria:  (a)  the  item 
had  a  mean  relevance  rating  of  4.00  or  higher  on  a  single  performance  dimension;  (b)  mean 
relevance  ratings  for  each  other  dimension  were  less  than  4.00;  and  (c)  the  difference  between 
the  highest  mean  relevance  rating  and  the  mean  relevance  ratings  for  all  other  dimensions  was  at 
least  0.50.  This  pattern  of  results  would  indicate  that  the  item  was  a  good  measure  of  knowledge 
relevant  to  one  performance  dimension  and  it  was  a  better  measure  for  that  dimension  than  for 
any  other  dimension.  This  decision  rule  yielded  five  items  targeted  at  decision  making/problem 
solving,  nine  items  targeted  at  maintaining  good  working  relationships,  eight  items  targeted  at 
organizational  commitment,  five  items  targeted  at  showing  initiative,  and  eight  items  targeted  at 
adapting  to  uncertain  or  changing  situations.  Because  of  the  small  number  of  items  targeted  at 
decision  making/problem  solving,  we  added  one  item  that  had  a  mean  relevance  rating  of  3.91  on 
the  target  dimension,  with  a  maximum  mean  rating  on  other  dimensions  of  3.00.  Thus,  the  final 
version  of  the  SJT  consisted  of  36  items. 

Respondents  were  asked  to  do  three  things  in  responding  to  each  item:  (a)  choose  the 
most  effective  response  to  the  situation,  (b)  choose  the  least  effective  response  to  the  situation, 
and  (c)  rate  the  effectiveness  of  each  response  option.  This  variety  of  responses  allowed  us  to 
experiment  with  many  alternative  methods  for  scoring  the  SJT  items.  The  SJT  is  included  in 
Appendix  J. 

Step  5:  Scoring  key  development.  In  addition  to  the  cadet  ratings  of  response  option 
effectiveness,  a  group  of  10  cadre  officers  from  the  ROTC  units  included  in  this  project  provided 
effectiveness  ratings  for  the  SJT  to  ensure  its  appropriateness  for  the  ROTC  and  to  provide  an 
alternative  source  of  SME  ratings  to  be  used  for  scoring  the  SJT.  The  cadre  officers  were  90% 
male,  came  from  different  universities  and  services  (4  Air  Force,  4  Army,  2  Navy),  and  held  a 
variety  of  officer  ranks.  After  examining  the  data,  a  male  Air  Force  CPT  and  a  male  Army  MAJ 
were  dropped  from  this  group.  The  former  exhibited  poor  agreement  with  the  other  officers  in 
the  sample,  while  the  latter  did  not  complete  64%  of  the  ratings,  and  those  he  did  complete 
exhibited  poor  agreement  with  the  other  officers  in  the  sample. 

Four  alternative  scoring  keys  were  developed  for  the  SJT  based  on  different  SME  groups: 
(a)  Cadre  Key  (N=  8),  (b)  Cadet  Key  (N-  15),  (c)  Combined  Cadre  +  Cadet  Key  (N=  23),  and 
(d)  a  Consensus-Based  Key  (N =  153).  The  latter  consensus-based  key  is  based  on  an  approach 
discussed  by  Legree  et  al.  (2005),  which  suggests  that  aggregating  the  responses  of  examinees 
may  be  advantageous  for  domains  lacking  certified  experts  and  where  objective  knowledge  is  ill- 
specified.  Mean  effectiveness  ratings  were  calculated  and  assigned  to  each  response  option. 
Because  these  effectiveness  ratings  differed  across  SME  groups,  four  different  scoring  keys  were 
created,  and  four  sets  of  scores  were  calculated  for  subjects.  A  score  for  each  item  was  computed 
by  subtracting  the  mean  effectiveness  rating  for  the  response  chosen  as  least  effective  from  the 
mean  effectiveness  rating  for  the  response  chosen  as  most  effective.  For  example,  if  an 
individual  rated  response  A  the  most  effective  and  C  the  least  effective,  and  response  A  had  a 


24 


mean  effectiveness  rating  of  5.0  and  response  C  had  a  mean  effectiveness  rating  of  3.5,  then  the 
individual’s  score  for  that  item  would  be  1.5.  Analyses  conducted  to  evaluate  the  alternative 
scoring  keys  are  detailed  in  the  Results  section. 

Past  Behavior  Record 

The  Past  Behavior  Record  (PBR)  was  developed  for  this  research  and  measures  skills 
relevant  to  each  of  the  five  target  performance  dimensions.  We  defined  skill  as  the  extent  to 
which  a  person  is  actually  able  to  perform  a  particular  behavior  relevant  to  a  targeted 
performance  dimension.  This  is  distinguished  from  the  actual  performance  of  the  behavior 
because  it  does  not  include  the  motivation  component  that  determines  whether  a  person  will 
actually  do  what  he  or  she  is  able  to  do.  For  example,  a  cadet  may  possess  the  skill  necessary  to 
cooperate  with  others,  but  could  exhibit  lower  performance  on  that  dimension  if  he  or  she  is 
infrequently  motivated  to  do  so.  We  measured  skill  relevant  to  the  targeted  performance 
dimensions  with  a  past  behavior  record.  This  is  similar  to  a  behavior  description  interview  (Janz, 
Hellervik,  &  Gilmore,  1 986),  but  in  a  written  format,  making  it  more  like  an  accomplishment 
record  (Hough,  1984).  Using  a  highly  structured  format  to  minimize  the  possible  confounding 
effect  of  writing  ability,  cadets  described  a  recent  situation  they  experienced  that  elicited 
behavior  relevant  to  the  performance  dimension  of  interest,  the  behavior  they  exhibited,  and  the 
outcome  of  their  behavior.  Trained  raters  used  standardized  rating  scales  to  evaluate  the  level  of 
skill  the  behavior  represented.  By  allowing  respondents  to  describe  their  best  example  of 
behavior  relevant  to  a  performance  dimension,  we  were  able  to  assess  the  extent  to  which  the 
person  exhibited  the  behavior  irrespective  of  the  extent  to  which  they  typically  exhibit  the 
behavior. 

We  used  a  similar  methodology  to  develop  the  PBR  that  we  have  used  successfully  in  the 
past  to  develop  accomplishment  records  (e.g.,  Bruskiewicz,  Johnson,  Lammlein,  &  Carter, 

2001).  First,  we  created  a  form  similar  to  the  example  form  provided  by  Hough  (1984;  p.  137), 
as  well  as  forms  that  are  typically  used  to  record  responses  to  behavior  description  interviews.  At 
the  top  of  the  page  we  provided  the  target  dimension  name  and  definition,  including  a  list  of 
behaviors  that  were  relevant  to  the  dimension.  The  form  asked  the  respondent  to  describe  (a)  a 
situation  from  the  respondent’s  recent  past  (preferably  ROTC-related)  that  required  the 
respondent  to  exhibit  behavior  relevant  to  the  target  dimension,  (b)  the  action  taken  by  the 
respondent  in  response  to  the  situation,  and  (c)  the  outcome  or  result  of  the  behavior.  Specific 
probes  within  each  of  these  three  domains  (situation-behavior-outcome)  were  included  to  keep 
respondents  focused  on  only  the  information  required.  The  PBR  is  included  in  Appendix  K. 

Next,  we  conducted  two  workshops  involving  12  cadets  and  midshipmen  of  varying 
levels  of  experience  to  collect  examples  of  past  behavior.  We  asked  participants  to  describe  one 
or  two  ROTC-relevant  behavioral  examples  from  their  own  past  for  each  dimension  using  the 
PBR  form.  By  restricting  behaviors  to  ROTC  situations  and  including  cadets  with  varying  levels 
of  experience,  we  obtained  a  set  of  behaviors  that  varied  in  their  level  of  effectiveness.  These 
statements  were  typed  and  edited  for  grammar  and  clarity. 

Third,  we  conducted  two  workshops  involving  1 0  fourth-year  cadets  and  midshipmen  in 
which  we  asked  the  participants  to  rate  each  behavioral  example  obtained  in  the  previous 


25 


workshop  on  its  level  of  effectiveness.  We  also  asked  why  each  rating  was  made  (i.e.,  what 
aspect  of  the  behavior  made  it  especially  effective,  ineffective,  or  moderately  effective?). 
Following  the  workshop,  we  computed  the  mean  effectiveness  rating  for  each  example  and 
ordered  them  within  dimension  according  to  effectiveness.  Two  PDRI  psychologists  then 
induced  the  principles  underlying  the  rating  process  by  identifying  the  underlying  themes  that 
examples  had  in  common  and  that  distinguished  between  high,  medium,  and  low  levels  of 
effectiveness.  They  used  these  principles  and  the  information  provided  by  workshop  participants 
to  develop  rating  scales  for  each  dimension  that  contained  behavioral  statements  describing  high, 
medium,  and  low  levels  of  effectiveness  for  each  facet  of  the  dimension.  Along  with  instructions 
for  making  ratings,  these  rating  scales  constituted  the  Evaluation  Guide  that  was  used  by  judges 
to  evaluate  cadets’  PBR  responses. 

Two  PBR  scoring  guides  were  developed:  (a)  a  conventional  version,  and  (b)  a  multi-part 
anchored  rating  scale  (MARS)  version  (Ferstl  &  Houston,  2006).  Anchor  facets  were  arranged 
differently  across  versions,  but  their  content  was  identical.  In  the  conventional  version,  anchor 
facets  were  grouped  by  level  (high,  medium,  low).  In  MARS,  they  were  grouped  by  facet  type 
(situation,  behavior,  and  outcome).  The  other  difference  between  the  versions  is  that  MARS  does 
not  ask  for  an  overall  numerical  rating;  raters  merely  check  the  best-fitting  anchor  within  each 
facet  type.  Raters  were  required  to  rate  situation,  behavior,  and  outcome  using  both  formats  so 
we  could  aggregate  ratings  across  raters,  regardless  of  the  format  used.  The  conventional 
evaluation  guide  is  included  in  Appendix  L  and  the  MARS  scoring  guide  is  included  in 
Appendix  M. 

PBR  judges  were  advanced  graduate  students.  Four  judges  were  trained  in  the  use  of  both 
scoring  guides  and  how  to  rate  PBR  responses.  We  conducted  pilot  work  in  which  judges  rated  a 
sample  of  the  example  behaviors.  We  evaluated  the  interrater  reliability  of  these  ratings  and  their 
correspondence  to  ratings  made  by  advanced  cadets  to  ensure  that  judges  were  able  to  make 
reliable  and  valid  ratings.  We  revised  the  scales  where  necessary  as  a  result  of  feedback  from  the 
pilot  work. 

After  data  collection,  each  judge  used  the  MARS  rating  scales  for  half  of  the  participants 
and  the  conventional  rating  scales  for  the  other  half.  Thus,  each  participant’s  PBR  was  rated  four 
times,  twice  using  each  type  of  rating  scale. 

Procedure 


In-Person  Predictor  Battery 

The  first  portion  of  the  predictor  battery  was  administered  during  the  Fall  2006  semester 
at  each  university.  Participants  signed  up  for  a  3-hour  group  session  to  complete  the  following 
instruments:  (a)  Wonderlic  Personnel  Test,  (b)  Citizenship  Motives  Scale,  (c)  Situational 
Judgment  Test,  (d)  Motivational  Trait  Questionnaire,  and  (e)  Past  Behavior  Record.  Participants 
were  paid  $50  for  completion  of  the  in-person  predictor  battery. 
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Online  Predictor  Battery 

After  completion  of  the  in-person  predictor  battery,  participants  received  instructions  to 
complete  six  additional  instruments  online.  Participants  also  received  these  instructions  via  email 
with  direct  links  to  the  survey  website.  The  online  battery  consisted  of  six  instruments:  (a)  the 
IPIP  Personality  Inventory,  (b)  Work  Habits  Scale,  (c)  Experience  Inventory,  (d)  Military 
Motives  Scale,  (e)  Proactive  Cognitions  Scale,  and  (f)  Action  Control  Scale.  The  estimated  time 
to  complete  all  six  brief  instruments  was  90  minutes,  and  participants  were  asked  to  complete  the 
instruments  within  two  weeks.  Non-respondents  were  given  up  to  three  reminder  emails  to 
complete  all  of  the  instruments.  Among  the  1 55  participants  who  completed  the  in-person 
battery,  136  (87.7%)  completed  all  six  of  the  online  inventories.  Participants  were  paid  $30  for 
completion  of  the  online  predictor  battery. 

Ongoing  Online  Assessment 

After  completion  of  the  online  predictor  battery,  participants  entered  a  phase  of  ongoing 
survey  administration.  Approximately  every  month  for  four  months,  participants  were  asked  to 
complete  the  1 5-minute  Self-Regulation  Inventory  online.  At  the  point  of  each  survey 
administration,  participants  were  sent  an  email  with  a  direct  link  to  the  survey.  Non-respondents 
received  a  maximum  of  two  reminders  to  complete  the  ongoing  surveys.  Response  rates 
generally  declined  over  the  course  of  the  four  ongoing  surveys:  there  was  a75.5%  response  rate 
for  the  first  survey,  a  65.8%  response  rate  for  the  second  survey,  a  55.5%  response  rate  for  the 
third  survey,  and  a  64.5%  response  rate  for  the  fourth  survey.  Participants  were  paid  $8  for 
completing  each  of  the  online  assessments,  with  an  $8  bonus  for  completing  all  four  assessments 
(total  of  $40  possible). 

Performance  Ratings  Administration 

In  the  middle  of  the  spring  2007  semester,  performance  ratings  were  collected  for  each 
participant  involved  in  the  predictor  administration.  The  performance  ratings  were  completed  by 
the  participants’  ROTC  peers  for  research  purposes  only;  no  ROTC  personnel  were  given  access 
to  the  data.  To  identify  appropriate  raters,  all  participants  were  asked  to  indicate  which  of  the 
other  participants  (within  university)  they  were  capable  of  rating  (i.e.,  regular  interaction  with 
that  person  and  adequate  opportunities  to  observe  their  performance).  Among  the  original  155 
participants,  75  individuals  served  as  raters,  and  an  additional  1 5  ROTC  cadets  and  midshipmen 
were  recruited  to  serve  exclusively  as  raters. 

Raters  were  matched  with  up  to  four  individuals  from  the  list  of  participants  they 
indicated  they  could  rate.  Rating  assignments  were  emailed  to  raters,  with  a  direct  link  to  the 
online  performance  assessment.  At  the  end  of  the  online  survey,  raters  indicated  whether  they 
were  willing  to  rate  additional  participants.  If  they  answered  yes  to  this  question,  they  were 
assigned  up  to  four  more  of  their  peers  to  rate  (depending  on  the  participants  they  were  capable 
of  rating  and  the  need  for  raters).  Collection  of  performance  ratings  continued  in  this  manner 
until  all  155  original  participants  had  at  least  one  performance  rating  (mean  =  3.27 
raters/participant;  SD  =  1 .00).  Raters  were  paid  $10  for  completing  one  performance  rating  and 
$5  for  each  additional  rating  completed. 
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Results 


Predictor  Descriptive  Statistics 

Means,  standard  deviations,  and  alphas  for  all  predictor  variables  are  presented  in  Table 
3.  For  the  personality  variables.  Big  Five  factor  scores  were  computed  by  calculating  the  mean 
item  score  across  all  facets  composing  each  factor  (Intellect  was  measured  by  a  single  facet). 
Emotional  stability  is  composed  of  anxiety  and  anger.  Extraversion  is  composed  of  friendliness, 
gregariousness,  and  assertiveness.  Agreeableness  is  composed  of  trust  and  cooperation. 
Conscientiousness  is  composed  of  self-efficacy,  dutifulness,  dependability,  and  initiative.  For  the 
MTQ  scales,  personal  mastery  is  composed  of  determination,  desire  to  learn,  and  mastery  goals. 
Competitive  excellence  is  composed  of  other-referenced  goals  and  competition  seeking. 
Motivation  anxiety  is  composed  of  worry,  emotionality,  and  evaluation  apprehension. 

Personality  items  were  recoded  where  necessary  so  that  higher  scores  meant  the  individual 
possesses  more  of  the  trait. 

Because  of  the  large  number  of  variables,  predictor  intercorrelations  are  presented  in  the 
appendices.  Appendix  N  presents  intercorrelations  between  the  predictor  variables  that  are 
relevant  for  all  performance  dimensions.  These  variables  include  the  WPT,  personality  scales, 
social  insight,  action  control  scales,  job  satisfaction,  military  values,  and  affective  commitment. 
We  measured  both  affective  and  continuance  commitment,  but  continuance  commitment  was 
uncorrelated  with  all  other  variables  in  the  investigation  so  we  did  not  include  this  scale  in  any 
data  analyses.  Appendix  O  presents  intercorrelations  between  dimension-specific  predictors. 
These  include  experience,  interests,  citizenship  motives,  expectancy,  efficacy,  commitment, 
habits,  PBR,  SJT,  and  self-regulation. 

Criterion  Descriptive  Statistics 

We  computed  internal  consistency  reliabilities  (alphas)  for  each  performance  dimension 
to  ensure  that  each  item  contributed  positively  to  the  measurement  of  its  intended  dimension. 

One  item  from  decision  making/problem  solving  (item  6)  was  dropped  because  it  was  not  highly 
correlated  with  the  other  items  and  its  presence  decreased  alpha.  Another  item  (item  7)  was 
moved  from  decision  making/problem  solving  to  maintaining  good  working  relationships 
because  it  decreased  alpha  for  decision  making/problem  solving,  increased  alpha  for  maintaining 
good  working  relationships,  and  the  content  was  better  suited  to  maintaining  good  working 
relationships.  The  item  content  for  all  other  dimensions  remained  as  intended.  To  create 
performance  dimension  scale  scores,  we  computed  the  mean  of  the  items  within  each 
performance  dimension.  The  mean  rating  for  each  ratee  across  raters  was  then  computed  to 
arrive  at  a  single  dimension  score  for  each  ratee.  Descriptive  statistics  for  performance  rating 
scales  are  presented  in  Table  4.  Intercorrelations  between  performance  rating  scales  are 
presented  in  Table  5. 
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Table  3 

Descriptive  Statistics  and  Alphas  for  Predictor  Variables 


Variable 

N 

Mean 

SD 

Min 

Max 

Alpha 

Wonderlic  Personnel  Test 

155 

27.08 

5.12 

13 

39 

— 

Big  5:  Emotional  Stability 

139 

3.55 

0.70 

1.80 

5.00 

.91 

Big  5:  Extraversion 

139 

3.77 

0.61 

2.20 

4.83 

.94 

Big  5:  Agreeableness 

139 

3.53 

0.63 

1.60 

4.95 

.89 

Big  5:  Conscientiousness 

139 

4.08 

0.38 

2.65 

4.89 

.91 

Big  5:  Intellect 

139 

3.80 

0.72 

1.80 

5.00 

.89 

Anxiety 

139 

2.50 

0.76 

1.00 

4.50 

.85 

Anger 

139 

2.40 

0.82 

1.00 

4.50 

.90 

Friendliness 

139 

3.84 

0.76 

1.40 

5.00 

.91 

Gregariousness 

139 

3.59 

0.75 

1.60 

5.00 

.88 

Assertiveness 

139 

3.89 

0.61 

2.40 

5.00 

.84 

Trust 

139 

3.72 

0.77 

1.00 

5.00 

.91 

Cooperation 

139 

3.35 

0.69 

1.50 

4.90 

.81 

Self-Efficacy 

138 

4.25 

0.48 

2.60 

5.00 

.86 

Dutifulness 

139 

4.31 

0.45 

2.57 

5.00 

.83 

Dependability 

139 

3.96 

0.53 

2.62 

5.00 

.81 

Initiative 

139 

3.71 

0.53 

1.44 

4.89 

.79 

Cautiousness 

139 

3.41 

0.56 

2.10 

4.80 

.77 

Hope/Optimism 

138 

3.98 

0.62 

2.25 

5.00 

.78 

V  alor/Bravery/Courage 

139 

3.79 

0.55 

2.60 

5.00 

.77 

Situational  Flexibility 

139 

3.18 

0.54 

1.83 

4.50 

.75 

Conflict  Resolution 

139 

3.34 

0.61 

1.83 

4.75 

.82 

MTQ:  Personal  Mastery 

155 

4.49 

0.49 

3.06 

5.85 

.66 

MTQ:  Determination 

155 

4.74 

0.48 

3.20 

6.00 

.79 

MTQ:  Desire  to  Learn 

155 

4.88 

0.51 

3.30 

6.00 

.84 

MTQ:  Mastery  Goals 

155 

3.98 

0.78 

1.44 

5.78 

.84 

MTQ:  Competitive  Excellence 

155 

4.60 

0.59 

3.09 

6.00 

.64 

MTQ:  Other  Referenced  Goals 

155 

4.10 

0.68 

2.30 

5.65 

.88 

MTQ:  Competition  Seeking 

155 

4.00 

0.76 

2.15 

5.69 

.87 

MTQ:  Motivation  Anxiety 

155 

4.20 

0.82 

2.00 

6.00 

.82 

MTQ:  Worry 

155 

3.09 

0.85 

1.00 

5.44 

.83 

MTQ:  Emotionality 

155 

3.02 

0.74 

1.27 

5.22 

.83 

MTQ:  Eval.  Apprehension 

155 

3.20 

0.98 

1.00 

6.00 

.79 

MTQ:  Failure  Avoidance 

155 

2.66 

1.04 

1.00 

5.83 

.81 

Social  Insight 

155 

3.41 

0.96 

1.00 

5.50 

.83 

ACS:  Hesitation 

155 

2.82 

0.65 

1.46 

5.00 

.71 

ACS:  Preoccupation 

139 

3.44 

0.70 

1.44 

5.00 

.70 

ACS:  Volatility 

141 

5.33 

2.14 

0.00 

8.00 

.57 

Job  Satisfaction 

141 

5.30 

2.12 

0.00 

8.00 

.90 

Military  Values 

141 

4.62 

1.41 

1.00 

6.00 

.90 

Affective  Commitment 

139 

3.74 

0.66 

1.53 

5.00 

.78 

(table  continues) 
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Table  3  (continued) 


Variable 

N 

Mean 

SD 

Min 

Max 

Alpha 

Experience-Decision  Making 

138 

3.61 

0.41 

2.13 

4.00 

.83 

Experience-Work  Relations 

138 

4.03 

0.76 

1.25 

5.00 

.81 

Experience-Org  Commitment 

139 

3.62 

0.40 

2.17 

4.00 

.78 

Experience-Showing  Initiative 

139 

3.57 

0.36 

2.11 

4.00 

.85 

Experience-Adapting 

139 

3.68 

0.36 

2.17 

4.00 

.91 

Interest  in  Decision  Making 

139 

3.23 

0.53 

1.71 

4.00 

.79 

Interest  in  Work  Relationships 

139 

3.44 

0.50 

2.00 

4.00 

.81 

Interest  in  Org  Commitment 

138 

4.19 

0.53 

2.33 

5.00 

.85 

Interest  in  Showing  Initiative 

138 

4.36 

0.49 

3.00 

5.00 

.85 

Interest  in  Adapting 

137 

4.48 

0.53 

3.00 

5.00 

.89 

MGWR-Org  Concern 

137 

4.08 

0.60 

1.86 

5.00 

.75 

MGWR-Prosocial  Values 

137 

4.14 

0.63 

2.29 

5.00 

.85 

MGWR-Impress  Management 

155 

4.08 

0.46 

2.90 

5.00 

.90 

OC-Organizational  Concern 

155 

3.95 

0.61 

1.90 

5.00 

.82 

OC-Prosocial  Values 

155 

2.73 

0.92 

1.00 

4.67 

.94 

OC-Impression  Management 

155 

4.00 

0.57 

2.40 

5.00 

.90 

Si-Organizational  Concern 

155 

3.48 

0.93 

1.00 

5.00 

.81 

Si-Prosocial  Values 

154 

2.78 

0.96 

1.00 

5.00 

.90 

Si-Impression  Management 

155 

4.05 

0.59 

1.90 

5.00 

.91 

Decision  Making-Expectancy 

155 

3.63 

0.80 

1.30 

5.00 

.92 

Maintain  Relations-Expectancy 

155 

2.66 

1.01 

1.00 

4.89 

.92 

Org  Commitment-Expectancy 

138 

4.48 

0.62 

1.00 

5.00 

.90 

Showing  Initiative-Expectancy 

138 

4.51 

0.62 

1.00 

5.00 

.91 

Adapting-Expectancy 

138 

4.59 

0.58 

1.00 

5.00 

.94 

Decision  Making-Efficacy 

138 

4.41 

0.66 

1.14 

5.00 

.87 

Working  Relationships- 
Efficacy 

138 

4.51 

0.64 

1.00 

5.00 

.85 

Org  Commitment-Efficacy 

138 

4.50 

0.51 

1.50 

5.00 

.82 

Showing  Initiative-Efficacy 

138 

4.57 

0.45 

1.44 

5.00 

.85 

Adapting-Efficacy 

138 

4.60 

0.44 

2.83 

5.00 

.91 

Decision  Making-Commitment 

138 

4.30 

0.58 

2.00 

5.00 

.84 

Work  Relations-Commitment 

138 

4.55 

0.52 

1.63 

5.00 

.83 

Org  Commitment- 
Commitment 

138 

4.50 

0.44 

3.00 

5.00 

.85 

Showing  Initiative- 
Commitment 

138 

4.57 

0.43 

2.78 

5.00 

.85 

Adapting-Commitment 

138 

4.59 

0.46 

2.50 

5.00 

.94 

Habits-Decision  Making 

138 

4.23 

0.59 

2.29 

5.00 

.81 

Habits-Working  Relationship 

136 

4.52 

0.57 

1.13 

5.00 

.89 

Habits-Org  Commitment 

135 

5.58 

0.98 

1.50 

7.00 

.92 

Habits-Showing  Initiative 

135 

5.96 

0.93 

1.67 

7.00 

.91 

Habits-Adaptive  Performance 

134 

5.94 

1.11 

1.50 

7.00 

.93 

(table  continues) 
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Table  3  (continued) 


Variable 

N 

Mean 

SD 

Min 

Max 

Alpha 

PBR:  Decision  Making 

134 

5.51 

1.19 

1.43 

7.00 

— 

PBR:  Working  Relationships 

134 

5.89 

1.01 

1.25 

7.00 

— 

PBR:  Org  Commitment 

154 

2.46 

0.42 

1.00 

3.00 

— 

PBR:  Showing  Initiative 

153 

2.60 

0.42 

1.00 

3.00 

— 

PBR:  Adapting 

153 

2.41 

0.36 

1.50 

3.00 

— 

SJT-Decision  Making 

154 

2.64 

0.36 

1.75 

3.00 

.60 

SJT-Working  Relationships 

153 

2.85 

0.27 

1.75 

3.00 

.60 

SJT-Org  Commitment 

155 

2.84 

0.61 

-1.33 

3.76 

.49 

SJT-Showing  Initiative 

155 

3.61 

0.53 

0.20 

4.41 

.39 

SJT-Adapting 

155 

3.14 

0.82 

-2.48 

4.07 

.55 

Self-reg:  Decision  Making  T1 

155 

2.95 

0.59 

-0.63 

3.80 

.92 

Self-reg:  Decision  Making  T2 

155 

2.75 

0.72 

-1.84 

3.73 

.93 

Self-reg:  Decision  Making  T3 

117 

3.55 

0.68 

1.69 

5.00 

.95 

Self-reg:  Decision  Making  T4 

101 

3.79 

0.69 

2.00 

5.00 

.92 

Self-reg:  Working  Relations  T1 

86 

3.76 

0.74 

1.81 

5.00 

.93 

Self-reg:  Working  Relations  T2 

100 

3.75 

0.66 

1.88 

5.00 

.93 

Self-reg:  Working  Relations  T3 

116 

3.76 

0.69 

1.94 

5.00 

.95 

Self-reg:  Working  Relations  T4 

101 

3.87 

0.65 

2.13 

5.00 

.94 

Self-reg:  Org  Commitment  T1 

86 

3.86 

0.72 

1.88 

5.00 

.94 

Self-reg:  Org  Commitment  T2 

100 

3.81 

0.71 

1.88 

5;00 

.94 

Self-reg:  Org  Commitment  T3 

116 

3.76 

0.73 

1.63 

5.00 

.95 

Self-reg:  Org  Commitment  T4 

102 

3.86 

0.70 

1.94 

5.00 

.94 

Self-reg:  Showing  Initiative  T1 

86 

3.89 

0.71 

1.81 

5.00 

.95 

Self-reg:  Showing  Initiative  T2 

99 

3.85 

0.71 

1.88 

5.00 

.95 

Self-reg:  Showing  Initiative  T3 

115 

3.81 

0.74 

1.75 

5.00 

.95 

Self-reg:  Showing  Initiative  T4 

101 

3.92 

0.70 

1.69 

5.00 

.92 

Self-reg:  Adapting  T1 

86 

3.89 

0.73 

1.44 

5.00 

.93 

Self-reg:  Adapting  T2 

99 

3.90 

0.63 

2.00 

5.00 

.94 

Self-reg:  Adapting  T3 

116 

3.74 

0.75 

1.73 

5.00 

.94 

Self-reg:  Adapting  T4 

102 

3.81 

0.75 

1.69 

5.00 

.94 

Table  4 

Descriptive  Statistics  for  Performance  Rating  Scales 

Variable 

Mean 

SD 

Min 

Max 

Decision  making/problem  solving 

3.69 

0.61 

2.00 

5.00 

Maintaining  good  working  relationships 

3.79 

0.62 

2.27 

5.00 

Organizational  commitment 

3.84 

0.63 

1.70 

5.00 

Showing  initiative 

3.48 

0.69 

1.29 

5.00 

Adapting  to  changing/uncertain  situations 

3.65 

0.61 

2.21 

5.00 

Overall  performance 

3.72 

0.80 

1.42 

5.00 

Note.  N=  155. 
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Table  5 

Intercorrelations  Between  Performance  Rating  Scales 


Variable 

1 

2 

3 

4 

5  6 

1. 

Decision  making/problem  solving 

1.00 

2. 

Maintaining  good  working  relationships 

.82 

1.00 

3. 

Organizational  commitment 

.85 

.82 

1.00 

4. 

Showing  initiative 

.81 

.77 

.83 

1.00 

5. 

Adapting  to  changing/uncertain  situations 

.89 

.81 

.84 

.81 

1.00 

6. 

Overall  performance 

.87 

.83 

.91 

.81 

.87  1.00 

Note.  N=  155.  All  correlations  are  significant  at p<  .001. 

We  used  generalizability  theory  to  estimate  the  interrater  reliability  of  each  performance 
dimension  scale  score.  The  design  was  (r  :p)x  i,  or  raters  nested  within  ratees  and  crossed  with 
items.  This  design  was  used  because  each  ratee  was  rated  by  a  unique  set  of  raters  on  the  same 
set  of  items.  We  were  most  interested  in  the  consistency  of  the  relative  ranking  of  persons  across 
conditions,  so  we  computed  G-coefficients  based  on  a  relative  definition  of  error  rather  than  an 
absolute  definition  of  error  (DeShon,  2002).  The  G-coefficients  were  .63  for  decision 
making/problem  solving,  .63  for  maintaining  good  working  relationships,  .64  for  organizational 
commitment,  .60  for  showing  initiative,  .58  for  adapting  to  uncertain  or  changing  situations,  and 
.70  for  overall  performance. 

Because  of  the  large  number  of  variables,  correlations  between  predictors  and  criteria  are 
presented  in  the  appendices.  Correlations  between  variables  that  are  relevant  to  decision 
making/problem  solving  are  presented  in  Appendix  P.  Correlations  between  variables  that  are 
relevant  to  maintaining  good  working  relationships  are  presented  in  Appendix  Q.  Correlations 
between  variables  that  are  relevant  to  organizational  commitment  are  presented  in  Appendix  R. 
Correlations  between  variables  that  are  relevant  to  showing  initiative  are  presented  in  Appendix 
S.  Correlations  between  variables  that  are  relevant  to  adapting  to  uncertain  or  changing  situations 
are  presented  in  Appendix  T. 


Evaluation  of  Experimental  Measures 

In  this  section,  we  discuss  the  scoring  procedures  and  present  construct  validity  evidence 
for  the  experimental  measures  created  for  this  research.  These  include  the  SJT,  the  PBR,  the 
Work  Habits  Scale,  and  the  Self-Regulation  Inventory.  Construct  validity  evidence  is  based  on 
correlations  with  theoretically  relevant  constructs  and  lack  of  correlation  with  irrelevant 
constructs. 

Situational  Judgment  Test 

Recall  that  we  had  four  alternative  sources  of  effectiveness  ratings  for  creating  the 
scoring  key:  (a)  cadre  key,  (b)  cadet  key,  (c)  combined  cadre  +  cadet  key,  and  (d)  consensus- 
based  key.  To  evaluate  the  effectiveness  of  the  alternative  keys,  we  computed  internal 
consistency  reliabilities  and  validities  for  each  scale  as  scored  using  the  different  keys.  SJTs 
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typically  have  low  internal  consistency,  primarily  because  the  complexity  of  situations  and 
responses  make  them  a  function  of  multiple  traits  and  abilities  (Chan  &  Schmitt,  2005). 
Nevertheless,  we  wanted  to  maximize  internal  consistency  because  we  needed  to  create  indicator 
variables  from  the  SJT  to  measure  the  knowledge  construct  in  our  structural  equation  models.  If 
the  indicators  are  not  correlated  with  each  other,  measurement  of  the  construct  would  be 
compromised. 

We  computed  SJT  scale  score  using  each  scoring  key,  computed  alphas,  and  correlated 
the  scores  with  their  target  performance  dimensions.  Results  are  displayed  in  Table  6.  Note  that 
there  were  no  cadet  ratings  of  effectiveness  for  two  items  because  they  were  added  to  the  item 
pool  after  the  cadet  ratings  were  collected.  Also,  one  item  was  excluded  from  the  adaptive 
performance  scale  because  it  had  a  large  detrimental  effect  on  both  reliability  and  validity. 

Several  things  are  noteworthy  from  this  table.  First,  validities  were  low  across  all  scoring 
keys  for  organizational  commitment  and  showing  initiative.  Second,  maintaining  good  working 
relationships  has  about  the  same  level  of  validity  across  all  scoring  keys.  Third,  the  validity  for 
decision  making/problem  solving  is  much  lower  for  the  consensus-based  key  than  for  the  others. 
Fourth,  validity  for  adaptive  performance  is  much  higher  for  the  cadre  key  than  for  the  others. 
Finally,  alphas  were  quite  low,  and  were  not  consistently  higher  for  one  scoring  key  than 
another.  We  chose  the  cadre  scoring  key  because  of  the  superior  validity  for  adaptive 
performance  and  decision  making/problem  solving,  but  that  still  left  a  problem  of  low  alphas. 

The  easiest  way  to  increase  internal  consistency  is  to  increase  the  number  of  construct¬ 
relevant  items  in  the  scale.  We  were  limited  in  the  number  of  items  we  could  include  targeting 
each  performance  dimension  because  of  testing  time,  but  one  way  to  increase  the  number  of 
items  is  to  allow  item  overlap  across  dimensions.  Our  analyses  were  conducted  within 
performance  dimension,  so  there  was  no  need  to  have  unique  items  for  each  SJT  scale. 

We  decided  to  use  a  construct-oriented  approach  that  takes  advantage  of  the  large  amount 
of  data  available  to  us  to  choose  items  for  each  SJT  scale.  We  used  the  following  sources  of 
information  to  create  SJT  scales:  (a)  construct  relevance  ratings  made  by  IO  psychologists,  (b) 
correlations  with  other  theoretically  relevant  predictors,  and  (c)  internal  consistency.  The 
potential  pool  of  items  for  a  given  scale  was  the  items  with  mean  construct  relevance  ratings  of 
3.00  or  higher,  regardless  of  what  the  mean  relevance  rating  was  for  any  other  performance 
dimension.  We  computed  correlations  between  each  item  score  and  other  variables  that  we 
expected  to  be  related  to  the  SJT  score.  For  example,  items  relevant  to  decision  making/problem 
solving  were  expected  to  be  primarily  correlated  with  the  WPT,  the  PBR,  and  experience,  with 
other  possible  correlations  with  the  self-efficacy  and  goal  commitment  proactive  cognitions 
scales  and  the  personality  traits  of  emotional  stability,  intellectance,  assertiveness,  and 
generalized  self-efficacy.  Items  that  showed  consistent  low  or  negative  correlations  with  these 
variables  were  considered  poor  measures  of  the  construct  and  were  removed  from  consideration. 
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Table  6 

Comparison  of  SJT  Scoring  Keys 
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Alphas  were  computed  for  scales  composed  of  the  remaining  items.  Items  that  decreased 
alpha  were  dropped  unless  they  had  very  strong  construct  validity.  Table  7  displays  the  alphas 
and  validities  for  the  revised  SJT  scales  using  the  cadre  scoring  key.  Alpha  increased 
considerably  for  decision  making/problem  solving,  which  originally  had  the  lowest  alpha  of  all 
the  scales.  Alphas  also  increased  for  maintaining  good  working  relationships  and  adaptive 
performance.  Alphas  were  unchanged  for  organizational  commitment  and  showing  initiative. 
Validities  for  these  dimensions  increased  but  were  still  at  a  low  level.  Validity  increased 
substantially  for  decision  making/problem  solving  and  slightly  for  maintaining  good  working 
relationships,  but  decreased  very  slightly  for  adaptive  performance.  We  concluded  that  the 
construct-oriented  approach  we  employed  is  a  useful  strategy  for  constructing  SJT  scales. 

Table  7 

Alphas  and  Validities  for  SJT  Scales  Developed  Using  Construct-Oriented  Approach 


Scale 

Items 

Alpha 

Validity 

Decision  making/problem  solving 

13 

.60 

.18* 

Maintaining  good  working  relationships 

14 

.60 

.18* 

Organizational  commitment 

6 

.49 

.08 

Showing  initiative 

8 

.39 

.03 

Adapting  to  uncertain/changing  situations 

8 

.55 

.17* 

Note.  N  =  155.  */><.05. 

Past  Behavior  Record 

Recall  that  we  created  two  different  scales  for  scoring  the  PBR:  (a)  a  conventional 
behavior  summary  scale,  and  (b)  a  multi-part  anchored  rating  scale  (MARS).  The  purpose  of 
creating  these  two  scales  was  to  compare  different  types  of  rating  scales,  which  will  be  the  focus 
of  a  later  project.  For  this  research,  we  needed  to  determine  what  single  score  across  rating  scales 
would  be  used  to  operationalize  skill  with  respect  to  each  performance  dimension.  The 
conventional  scale  required  the  rater  to  make  an  overall  rating  in  addition  to  ratings  of  the 
situation,  behavior,  and  outcome.  Correlations  between  these  overall  ratings  and  their  target 
performance  dimensions  ranged  from  -.03  to  .13,  with  none  being  significant.  Examining  the 
correlations  between  the  target  performance  dimensions  and  the  situation,  behavior,  and  outcome 
ratings  showed  that  many  of  these  relationships  were  significant.  This  suggests  that  the  clinical 
method  these  raters  used  to  arrive  at  an  overall  judgment  was  not  a  particularly  valid  procedure. 

The  MARS  scale  requires  raters  to  make  only  situation,  behavior,  and  outcome  ratings 
and  then  these  ratings  are  combined  using  a  standardized  scoring  method  to  create  the  overall 
score.  We  found  that  the  standard  scoring  procedure  typically  used  with  MARS  (giving  behavior 
twice  the  weight  as  situation  or  outcome;  Ferstl  &  Houston,  2006)  did  not  apply  very  well  to 
each  of  the  performance  dimensions  included  in  this  research.  For  some  dimensions,  behavior 
was  the  primary  predictor  of  performance,  but  for  other  dimensions,  situation  was  a  better 
predictor.  In  some  cases,  the  rated  difficulty  of  the  situation  on  the  PBR  was  highly  negatively 
correlated  with  performance  on  the  dimension,  which  was  not  expected.  Ordinarily,  successfully 
overcoming  a  more  difficult  situation  would  indicate  a  higher  level  of  skill,  so  one  would  expect 
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a  positive  correlation  between  situation  difficulty  and  performance.  As  an  example  of  this 
unexpected  result,  difficulty  of  the  situation  was  negatively  correlated  with  organizational 
commitment  performance.  Examining  the  situation  question  on  the  PBR  form,  we  determined 
that  this  made  sense  because  someone  who  was  very  committed  would  tend  not  to  encounter 
situations  in  which  it  is  difficult  to  maintain  his  or  her  commitment. 

We  also  conducted  multiple  regression  analyses  to  create  a  scoring  algorithm  for  each 
dimension,  in  which  the  algorithm  was  determined  based  on  the  mean  ratings  for  two  raters  and 
cross-validated  against  the  mean  ratings  for  the  other  two  raters.  This  yielded  valid  scoring 
algorithms,  but  we  did  not  believe  this  was  giving  us  what  we  needed  for  this  research.  The  goal 
was  to  measure  skill  relevant  to  each  performance  dimension,  not  to  predict  each  performance 
dimension.  We  had  to  allow  for  the  possibility  that  skill  may  not  be  a  relevant  construct  for  a 
performance  dimension  like  organizational  commitment,  which  was  only  predictable  by  the 
difficulty  of  the  situation.  The  difficulty  of  the  situation  is  not  a  measure  of  skill  regardless  of 
how  strong  the  correlation  is  with  performance.  Therefore,  we  determined  that  the  best  measure 
of  skill  was  the  behavior  exhibited  and  used  the  mean  behavior  rating  across  all  four  raters  as  the 
measure  of  skill  for  this  research. 

We  computed  interrater  reliability  using  ICC(2,  k),  which  assesses  the  level  of  absolute 
agreement  for  a  mean  of  the  ratings  across  raters  (Shrout  &  Fleiss,  1979).  Interrater  reliabilities 
were  .75  for  decision  making/problem  solving,  .78  for  maintaining  good  working  relationships, 
.63  for  organizational  commitment,  .70  for  showing  initiative,  and  .66  for  adapting  to  uncertain 
or  changing  situations. 

Self-Regulation  Inventory 

The  Self-Regulation  Inventory  (SRI)  was  designed  to  measure  the  extent  to  which 
participants  had  difficulty  in  implementing  their  intention  to  behave  in  a  certain  way.  It  consisted 
of  17  items  that  were  applied  to  each  of  the  five  performance  dimensions.  The  SRI  was 
administered  four  times  over  the  course  of  four  months,  with  the  idea  that  multiple 
measurements  would  capture  the  typical  level  of  self-regulation  a  participant  experienced  over 
the  evaluation  period. 

Out  of  the  155  cadets  and  midshipmen  who  completed  the  in-person  test  battery,  31  did 
not  respond  to  any  of  the  invitations  to  complete  the  SRI.  Twelve  participants  completed  it  once, 
14  completed  it  twice,  28  completed  it  three  times,  and  70  completed  all  four  surveys.  The  mean 
number  of  surveys  completed  was  2.61  (SD  =  1.59).  Table  8  shows  correlations  between 
performance  and  mean  SRI  scores  across  administrations  for  each  performance  dimension  for 
participants  who  completed  all  four  surveys,  then  when  adding  participants  who  completed  only 
three,  then  when  adding  participants  who  completed  only  two,  and  finally  for  all  participants.  As 
expected,  correlations  decrease  as  participants  are  added  for  whom  a  smaller  number  of  data 
points  are  available.  This  pattern  is  consistent  up  until  only  participants  for  whom  data  were 
available  for  all  administrations  were  included,  when  correlations  decreased.  These  results 
demonstrate  that  averaging  across  multiple  measurements  provides  better  prediction  than  using 
just  a  single  administration  of  this  type  of  survey. 
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Table  8 

Correlations  Between  Performance  and  Self-Regulation  When  Participants  Who  Completed 
Different  Numbers  of  Surveys  Are  Included  in  the  Sample 


Number  of  Surveys  Completed  (k) 
k  = 4  k> 3  k> 2  k=> 1 


Dimension 

II 

o 

(N  =  98) 

(N=  112) 

(N=  124) 

Decision  making/problem  solving 

.20 

.25* 

.24* 

.19* 

Maintaining  good  working  relationships 

.12 

.21* 

.18 

.14 

Organizational  commitment 

.19 

.21* 

.19* 

.17 

Showing  initiative 

.20 

.26** 

.20* 

.15 

Adapting  to  uncertain/changing  situations 

.17 

.19 

.17 

.15 

Note.  *p  <  .05.  **p  <  .01 . 

Table  9  shows  correlations  between  performance  and  self-regulation  for  each 
administration  of  the  survey.  Correlations  tended  to  increase  from  Time  1  to  Time  3,  which 
would  be  expected  because  measurements  that  are  more  proximal  to  when  performance  was 
measured  should  be  more  related  to  performance.  Correlations  drop  off  substantially  at  Time  4, 
however,  which  is  opposite  of  what  would  be  expected.  This  probably  explains  why  the 
correlations  in  Table  8  decrease  when  limited  to  those  who  completed  all  four  administrations, 
because  this  would  include  everyone  who  completed  the  survey  at  Time  4.  A  possible 
explanation  for  the  low  correlations  at  Time  4  is  that  the  administration  of  the  SRI  and  the 
collection  of  the  performance  ratings  occurred  at  approximately  the  same  time.  Perhaps  many  of 
the  raters  did  not  have  the  opportunity  to  observe  the  situations  rated  by  the  respondents  because 
they  occurred  too  recently. 

Table  9 

Correlations  Between  Performance  and  Self-Regulation  for  Each  Survey  Administration 


Dimension 

1 

(N=  117) 

Survey  Administration 

2  3 

(N=  101)  (N=  86) 

o 

o 

Decision  making/problem  solving 

.13 

.26** 

.32** 

.11 

Maintaining  good  working  relationships 

.02 

.19 

.22* 

.15 

Organizational  commitment 

.14 

.13 

.30** 

.08 

Showing  initiative 

.15 

.20 

.26* 

.10 

Adapting  to  uncertain/changing  situations 

.17 

.10 

.19 

.15 

Note.  *p  <  .05.  **p  <  .01. 

Table  9  shows  that  the  SRI  was  a  good  predictor  of  performance,  especially  at  Times  2 
and  3.  Additional  construct  validity  evidence  for  the  SRI  comes  from  correlations  with 
theoretically  relevant  predictors.  The  MTQ  scales  and  the  ACS  scales  should  predict  success  at 
self-regulating.  Table  10  presents  correlations  between  these  scales  and  the  mean  self-regulation 
score  across  administrations  for  each  performance  dimension. 
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Table  10 

Correlations  Between  Mean  Self-Regulation  Scores  and  Theoretically  Relevant  Predictors 


Self-Regulation  Scale  (Mean  Across  Administrations) 


Variable 

DMPS 
(N=  124) 

MGWR 
(N=  123) 

OC 

(N=  122) 

SI 

(N=  122) 

AUCS 
( N=  122) 

Motivational  Trait 

Personal  Mastery 

.31*** 

.25** 

.30*** 

32*** 

3  ]  *** 

Determination 

42*** 

.37*** 

41  *** 

4]  *** 

47*** 

Desire  to  Learn 

.12 

.09 

.12 

.15 

.12 

Mastery  Goals 

.26** 

.19* 

.25** 

.26** 

.23* 

Competitive  Excellence 

-.10 

-.18* 

-.09 

-.10 

-.18* 

Other-Referenced  Goals 

-.26** 

.  29*** 

-.23** 

-.25** 

_  3]*** 

Competition  Seeking 

.07 

-.03 

.06 

.07 

-.02 

Motivation  Anxiety 

_ 39*** 

.  44*** 

. 45*** 

-.44*** 

Worry 

-.38*** 

-39*** 

. 43*** 

-.42*** 

-  42*** 

Emotionality 

_38*** 

-.37*** 

_  27*** 

-.42*** 

_  39*** 

Evaluation  Apprehension 

_  2q*** 

-.26** 

-.35*** 

-  32*** 

. 35*** 

Failure  Aversion 

-.37*** 

-.38*** 

_ 42*** 

_  43*** 

-.50*** 

Action  Control  Scale 

Hesitation 

24*** 

.27** 

32*** 

.34*** 

35*** 

Preoccupation 

39*** 

39*** 

40*** 

42*  *  * 

Volatility 

.ii 

.10 

.ii 

.17 

.17 

Note.  DMPS  =  Decision  Making/Problem  Solving,  MGWR  =  Maintaining  Good  Working 
Relationships,  OC  =  Organizational  Commitment,  SI  =  Showing  Initiative,  AUCS  =  Adapting  to 
Uncertain  or  Changing  Situations.  *p  <  .05.  **p  <  .01.  ***p  <  .001. 

Many  of  the  MTQ  and  ACS  scales  had  very  strong  and  significant  correlations  with  the 
self-regulation  scales.  Note  that  the  ACS  scales  are  scored  such  that  high  scores  mean  the 
respondent  has  more  action  control  and  is  better  able  to  self-regulate,  so  these  correlations  are  in 
the  hypothesized  direction.  These  correlations  show  that  most  theoretically  relevant  variables  are 
highly  correlated  with  the  self-regulation  scales,  providing  strong  construct  validity  evidence  for 
the  SRI.  Correlations  between  these  theoretically  relevant  predictors  and  the  self-regulation 
scales  at  each  administration  time  can  be  found  in  Appendices  P  through  T.  The  level  of 
correlations  across  administrations  was  consistent  across  administrations,  indicating  that  the  SRI 
was  equally  effective  as  a  measure  of  self-regulation  success  at  each  administration.  This  gives 
further  evidence  to  the  possibility  that  low  correlations  with  performance  at  Time  4  are  due  to 
raters’  inability  to  observe  that  performance  rather  than  something  different  with  the  SRI  at  that 
administration. 
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Work  Habits  Scale 


According  to  the  Johnson  (2003)  model,  work  habits  should  be  directly  related  to  self¬ 
regulation  and  performance.  Habits  should  be  directly  related  to  performance  when  performance 
is  automatic,  and  should  be  related  to  self-regulation  when  habits  interfere  with  successful 
performance.  Another  variable  that  should  be  related  to  habits  is  experience,  because  one 
prerequisite  for  a  behavior  becoming  a  habit  is  that  it  must  be  performed  frequently.  Table  1 1 
presents  correlations  for  habits  with  performance,  self-regulation,  and  experience. 

Table  11 

Correlations  for  Habits  with  Performance,  Self-Regulation,  and  Experience 


Habits 

Performance 

Self-Regulation 

Experience 

Decision  making/problem  solving 

-.06 

.22* 

.23** 

Maintaining  good  working  relationships 

.04 

.20* 

39*** 

Organizational  commitment 

.12 

.13 

.22* 

Showing  initiative 

-.03 

.17 

.20* 

Adapting  to  uncertain/changing  situations 

-.01 

.28** 

.25** 

Note.  Pairwise  deletion  of  missing  cases.  Ns  range  from  1 18  to  135.  *p  <  .05.  **p  <  .01. 

***/?<  .001. 

Habits  were  significantly  correlated  with  experience  for  all  dimensions.  Habits  were 
significantly  correlated  with  self-regulation  for  only  three  dimensions,  however,  and  were 
uncorrelated  with  performance  for  all  dimensions.  This  calls  the  construct  validity  of  the  Work 
Habits  Scale  into  question.  To  further  explore  the  nature  of  the  Work  Habits  Scale,  we 
formulated  hypotheses  about  the  relationships  between  habits  and  interests,  expectancies,  self- 
efficacy,  and  goal  commitment.  We  anticipated  that  interests  would  be  good  predictors  of  habits 
because  people  tend  to  make  a  habit  out  of  doing  things  in  which  they  are  interested.  Habits 
should  predict  expectancies  because  something  becomes  a  habit  when  it  is  rewarded.  Therefore, 
a  person  whose  habits  are  consistent  with  good  performance  would  understand  that  the  behavior 
leads  to  desired  outcomes.  Habits  should  predict  self-efficacy  because  a  person  knows  that  he  or 
she  is  able  to  do  what  is  habitual.  Habits  should  predict  goal  commitment  because  when 
something  is  a  habit  it  is  very  easy  to  be  committed  to  do  it.  Table  12  presents  correlations  for 
habits  with  interests,  expectancies,  self-efficacy,  and  goal  commitment.  All  of  these  correlations 
were  significant  and  in  the  expected  direction. 
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Table  12 

Correlations  for  Habits  with  Interests,  Expectancies,  Self-Efficacy,  and  Goal  Commitment 


Habits 

Interests 

Expectancies 

Self- 

Efficacy 

Goal 

Commitment 

Decision  making/problem  solving 
Maintaining  good  working 

.38*** 

.18* 

49*** 

54*** 

relationships 

5 1  *** 

.26** 

.57*** 

.51*** 

Organizational  commitment 

4g*** 

42*** 

.55*** 

53*** 

Showing  initiative 

Adapting  to  uncertain/changing 

5  ]  *** 

.21* 

42*** 

40*** 

situations 

.36*** 

.20* 

47*** 

.37*** 

Note.  Pairwise  deletion  of  missing  cases.  Ns  range  from  130  to  134.  *p  <  .05.  **p  <  .01. 

***p<  .001. 

These  results  provide  construct  validity  evidence  for  the  Work  Habits  Scale.  It  may  be 
that  the  kinds  of  behaviors  assessed  in  this  project  are  not  likely  to  occur  automatically,  such  that 
they  would  bypass  the  motivation  component.  This  pattern  of  correlations  clearly  shows  that 
habits  are  not  direct  determinants  of  performance  on  these  dimensions,  but  may  have  an 
influence  on  motivation  to  perform  these  behaviors.  Habits  may  be  more  likely  to  have  a  direct 
effect  when  the  measurement  of  performance  is  on  simpler  or  more  specific  behaviors  in  specific 
situations. 


Importance  of  Performance  Dimensions  to  Construct  Domain 

We  wanted  to  statistically  determine  whether  the  five  performance  dimensions  chosen  for 
this  research  were  meaningful  in  describing  ROTC  cadet/midshipman  performance.  Model 
testing  would  be  irrelevant  for  a  performance  dimension  that  was  not  an  important  part  of  the 
performance  domain.  We  used  two  methods  to  test  the  importance  of  each  performance 
dimension:  (a)  testing  the  unique  contribution  to  ratings  of  overall  performance,  and  (b)  testing 
the  relative  importance  of  each  dimension  to  ratings  of  overall  performance. 

Unique  Contribution  to  Overall  Performance 

One  way  to  demonstrate  that  each  performance  dimension  is  an  important  part  of  the 
ROTC  cadet  performance  domain  is  testing  whether  each  performance  dimension  contributes 
uniquely  to  ratings  of  overall  job  performance.  Motowidlo  and  Van  Scotter  (1994),  Van  Scotter 
and  Motowidlo  (1996),  and  Conway  (1999)  used  this  approach  to  distinguish  between  task 
performance  and  citizenship  performance.  Because  adapting  to  uncertain  or  changing  situations 
also  is  considered  to  be  conceptually  distinct  from  task  and  citizenship  performance  (Campbell, 
1999;  Hesketh  &  Neal,  1999;  Johnson,  2003),  we  expected  that  each  performance  dimension 
would  contribute  unique  variance  to  ratings  of  overall  performance. 

To  control  for  halo  when  evaluating  the  contribution  of  task  and  citizenship  performance 
to  overall  performance,  Motowidlo  and  Van  Scotter  (1994)  and  Van  Scotter  and  Motowidlo 
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(1996)  had  a  different  rater  rate  each  performance  dimension  and  overall  performance.  We  used 
a  similar  procedure,  but  there  were  not  enough  raters  to  have  a  separate  rater  for  each 
performance  dimension.  Therefore,  we  used  a  hierarchical  regression  procedure  to  determine  if 
the  target  performance  dimension  contributed  significantly  to  the  prediction  of  overall 
performance  beyond  the  remaining  dimensions.  In  Step  1 ,  all  performance  dimensions  except  the 
target  dimension,  as  rated  by  a  single  rater  (e.g.,  Rater  1),  were  entered.  The  target  dimension,  as 
rated  by  another  rater  (e.g.,  Rater  2),  was  entered  in  Step  2.  Overall  performance  was  rated  by  the 
third  rater  (e.g.,  Rater  3).  Unlike  the  Motowidlo  and  Van  Scotter  studies,  all  raters  rated  all 
dimensions  in  this  project  so  there  were  six  different  combinations  of  rater  assignments.  Rather 
than  choosing  one  combination,  we  computed  the  correlation  matrices  for  all  combinations, 
computed  the  mean  correlations,  and  used  the  mean  correlation  matrix  to  perform  the 
hierarchical  regression  analysis.  This  was  done  five  times,  once  for  each  performance  dimension 
as  the  target  dimension. 

Results  are  presented  in  Table  13.  Each  performance  dimension  contributed  significant 
variance  beyond  the  other  performance  dimensions  when  predicting  overall  performance, 
supporting  the  hypothesis  that  each  performance  dimension  measured  a  unique  aspect  of  the 
performance  domain. 

Relative  Importance  to  Overall  Performance 

We  expected  that  each  performance  dimension  would  be  a  relatively  important 
contributor  to  the  evaluation  of  overall  performance  when  other  performance  dimensions  were 
included  in  the  regression  model.  To  evaluate  the  relative  importance  of  performance 
dimensions,  a  multiple  regression  analysis  was  conducted  with  all  performance  dimensions 
entered  simultaneously  and  overall  performance  as  the  dependent  variable.  Because  regression 
coefficients  are  not  interpretable  as  measures  of  relative  importance  when  independent  variables 
are  highly  correlated  (Darlington,  1968),  a  relative  weight  analysis  (Johnson,  2000)  was 
conducted.  Relative  weight  analysis  is  a  procedure  for  quantifying  the  relative  importance  of 
predictor  variables  in  multiple  regressions.  Relative  weights  reflect  the  proportionate 
contribution  each  predictor  makes  to  R2,  considering  both  its  unique  contribution  and  its 
contribution  when  combined  with  other  variables.  Relative  weights  better  reflect  the  manner  by 
which  raters  weight  and  combine  information  to  arrive  at  an  overall  performance  judgment, 
because  the  variance  explained  is  distributed  more  proportionately  among  the  predictors. 

Johnson  (2001)  used  this  procedure  to  demonstrate  the  relative  importance  of  specific  task  and 
citizenship  performance  dimensions  to  evaluations  of  overall  performance.  Relative  weights  sum 
to  R2  (just  as  squared  standardized  regression  coefficients  do  when  predictors  are  uncorrelated), 
so  they  are  expressed  as  percentages  of  the  predictable  variance  to  enhance  interpretation. 

Because  they  are  interpretable  when  predictors  are  highly  intercorrelated,  we  used  the  correlation 
matrix  in  Table  5  to  conduct  the  analysis. 
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Table  13 

Hierarchical  Regression  Results  When  Predicting  Overall  Performance 


Decision  making/problem  solving 

Step  Variables  entered 

R2 

A R2 

t 

E 

1  AUCS,  MGWR,  OC,  SI 

.240 

2  DMPS 

.294 

.054 

2.87 

.005 

Maintaining  good  working  relationships 

Step  Variables  entered 

R2 

A R2 

t 

E 

1  DMPS,  MGWR,  OC,  SI 

.225 

2  AUCS 

.295 

.070 

3.27 

.001 

Organizational  commitment 

Step  Variables  entered 

R2 

A R2 

t 

P 

1  DMPS,  MGWR,  OC,  SI 

.222 

2  AUCS 

.303 

.081 

3.53 

.001 

Showing  initiative 

Step  Variables  entered 

R2 

A R1 

t 

p 

1  AUCS,  MGWR,  OC,  SI 

.240 

2  DMPS 

.290 

.050 

2.76 

.007 

Adapting  to  uncertain  or  changing  situations 

Step  Variables  entered 

R2 

A R2 

t 

P 

1  DMPS,  MGWR,  OC,  SI 

.240 

2  AUCS 

.291 

.051 

2.78 

.006 

Note.  N=  114.  Regressions  were  conducted  on  mean  correlation  matrices  in  which  all  possible 
three-rater  combinations  were  averaged.  Rater  1  rated  the  performance  dimensions  entered  in 
Step  1 ;  Rater  2  rated  the  performance  dimension  added  in  Step  2;  Rater  3  rated  overall 
performance. 

Relative  weights  are  presented  in  Table  14.  Each  performance  dimension  contributed  a 
substantial  amount  to  the  prediction  of  overall  performance,  indicating  that  raters  tend  to 
consider  performance  on  all  five  dimensions  when  evaluating  overall  performance.  The  most 
important  dimension  was  organizational  commitment,  accounting  for  24.7%  of  the  predictable 
variance.  Adapting  to  uncertain  or  changing  situations  was  the  next  most  important  at  20.5%. 
Therefore,  the  hypothesis  that  each  performance  dimension  would  be  a  relatively  important 
predictor  of  overall  performance  ratings  was  supported. 
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Table  14 

Relative  Weight  Analysis  Results  When  Predicting  Overall  Performance 


Variable 

Relative  Weight 

%  of  R1 

Decision  making/problem  solving 

.174 

20.0% 

Maintaining  good  working  relationships 

.159 

18.2% 

Organizational  commitment 

.216 

24.7% 

Showing  initiative 

.145 

16.5% 

Adapting  to  uncertain/changing  situations 

.180 

20.5% 

Note.  N  =  155./^  =  874. 


Formulation  of  Structural  Equation  Models 

Structural  equation  models  were  tested  using  SPSS  Amos  7.0.  The  same  general 
procedure  was  used  to  build  and  test  models  for  each  performance  dimension.  We  describe  this 
general  procedure  in  the  following  sections,  and  then  present  the  specific  results  for  each 
performance  dimension. 

Identifying  Constructs  to  Include  in  Each  Model 

Because  of  the  large  number  of  indirect  performance  determinants  (i.e.,  exogenous 
variables)  included  in  this  research,  it  would  be  impossible  to  include  all  potentially  relevant 
variables  in  a  model.  Jackson  (2003)  indicated  that  the  sample  size  necessary  in  structural 
equation  modeling  is  related  to  the  number  of  parameters  estimated  in  the  model.  With  a  large 
number  of  parameter  estimates,  our  sample  size  would  be  too  small  to  have  confidence  in  the 
results.  Even  with  a  larger  sample  size,  the  model  tested  would  be  prohibitively  complex  and 
unlikely  to  cross  validate  in  another  sample.  To  minimize  the  number  of  parameters  to  be 
estimated  and  ensure  that  only  relevant  variables  were  included  in  structural  equation  models,  we 
examined  the  correlation  matrix  between  indirect  performance  determinants  (i.e.,  measures  of 
personality,  social  intelligence,  action  control,  experience,  and  general  cognitive  ability)  and 
each  performance  dimension.  Because  the  general  model  to  be  tested  posited  that  the  relationship 
between  indirect  performance  determinants  and  job  performance  is  mediated  by  knowledge, 
skill,  habits,  and  the  different  components  of  motivation,  we  excluded  measures  of  indirect 
performance  determinants  that  did  not  have  a  relationship  with  any  performance  dimension.  It 
would  be  meaningless  to  test  a  mediating  model  for  variables  that  are  not  correlated  with 
performance.  We  sought  to  explain  relationships  between  variables  with  a  mediating  model,  so 
we  included  only  indirect  performance  determinants  that  had  a  significant  relationship  with 
performance  (with  two  exceptions,  discussed  in  the  sections  describing  each  model  tested). 

After  identifying  indirect  performance  determinants  that  are  related  to  the  performance 
dimension  of  interest,  we  chose  three  or  four  to  include  in  the  model.  This  choice  was  based  on 
conceptual  relevance  to  the  performance  dimension  and  overlap  with  other  potential  variables. 

All  models  included  measures  of  the  mediating  variables  included  in  the  Johnson  (2003)  model 
(see  Figure  1 ).  How  these  constructs  were  operationalized  is  described  in  the  following  section. 
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Creating  Indicators  for  Latent  Variables 

Each  construct  is  measured  by  a  set  of  observed  variables  known  as  indicators.  When  the 
construct  is  by  definition  composed  of  more  specific  constructs  for  which  measures  are  available, 
those  specific  construct  measures  were  used  as  the  indicator  variables.  For  example,  the 
indicators  for  the  Agreeableness  construct  were  the  measures  of  trust  and  cooperation.  The 
indicators  for  the  Self-Regulation  construct  were  the  measures  of  self-regulation  at  Time  1,  Time 
2,  Time  3,  and  Time  4. 

When  a  construct  is  not  defined  by  a  more  specific  construct,  individual  items  could  be 
used  as  indicators.  This  would  create  a  very  large  number  of  parameters  to  be  estimated, 
however,  making  it  almost  impossible  to  fit  an  adequate  measurement  model.  According  to 
Landis,  Beal,  and  Tesluk  (2000),  creating  composites  of  indicator  variables  results  in  better 
model  fit  than  using  item-level  data  and  reduces  the  number  of  parameters  estimated.  There  are 
many  different  ways  to  create  composite  indicator  variables.  Three  potential  methods  of  creating 
composites  for  indicators  are  by  assigning  items  to  composites  (a)  randomly,  (b)  based  on  their 
intercorrelations,  or  (c)  based  on  their  factor  loadings  when  a  single  factor  is  extracted.  In  a 
Monte  Carlo  study,  Landis  et  al.  found  that  all  three  methods  provided  equally  good  model  fit 
and  parameter  estimates. 

We  used  the  single-factor  method  to  create  composite  indicator  variables.  In  the  single¬ 
factor  method,  a  principal  components  analysis  is  conducted  on  all  items  within  the  scale, 
extracting  one  factor.  If  two-item  composites  are  desired,  the  first  composite  is  created  by  pairing 
the  item  with  the  highest  loading  on  the  factor  with  the  item  with  the  lowest  loading  on  the  factor 
(Landis  et  al.,  2000).  This  process  is  repeated  with  the  remaining  items  until  all  items  have  been 
assigned  to  a  composite.  If  a  three-item  composite  is  created,  an  item  from  the  middle  of  the 
factor  loading  distribution  is  added  to  the  first  two  items.  If  a  four-item  composite  is  created,  the 
items  with  the  two  highest  loadings  are  combined  with  the  items  with  the  two  lowest  loadings. 
When  using  this  procedure,  we  created  three  indicator  variable  composites  for  each  construct. 

The  WPT  score  is  not  amenable  to  breaking  into  more  than  one  element,  so  we  used  the 
WPT  as  a  single  indicator  of  cognitive  ability.  Amos  has  no  way  of  estimating  the  error  variance 
when  there  is  only  one  indicator,  so  we  provided  Amos  with  an  estimate  of  the  error  variance 
using  the  following  formula: 


=°2(l~rxx)  0) 

where  cf  is  the  variance  of  the  WPT  and  /*»  is  its  reliability.  The  WPT  manual  reports  several 
reliability  estimates.  We  chose  the  Kuder-Richardson  20  value  of  .88  because  it  represents 
internal  consistency,  which  is  most  consistent  with  how  Amos  estimates  error  variance  for  latent 
variables  with  multiple  indicators. 

Table  15  lists  the  indicator  variables  for  each  construct  that  was  included  in  any  of  the 
models. 
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Table  15 

Indicator  Variables  for  Each  Construct  Included  in  Structural  Equation  Models 


Construct 

Models 

Indicators 

Performance 

All 

Three  indicators  created  from  performance  rating 
items 

Knowledge 

All 

Three  indicators  created  from  SJT  items 

Skill 

All  except  OC 

Two  indicators,  each  the  mean  of  one  rating  from 
conventional  scale  and  one  rating  from  MARS  scale 

Self-Regulation 

All 

Self-regulation  measure  at  Time  1,  Time  2,  Time  3, 

Time  4 

Habits 

All 

Three  indicators  created  from  habit  items 

Expectancies 

All 

Three  indicators  created  from  expectancy  items 

Goal  Commitment 

DMPS,  MGWR, 
AUCS 

Three  indicators  created  from  goal  commitment  items 

Military  Motives 

All 

Job  satisfaction,  military  values,  affective  commitment 

Interests 

All 

Three  indicators  created  from  interest  items 

Experience 

All 

Three  indicators  created  from  experience  items 

Cognitive  Ability 

All 

Wonderlic  Personnel  Test 

Conscientiousness 

DMPS,  OC,  SI 

Self-efficacy,  dutifulness,  dependability,  initiative 

Emotional  Stability 

DMPS 

Anxiety,  anger 

Intellectance 

DMPS,  SI, 

AUCS 

Three  indicators  created  from  intellectance  items 

Agreeableness 

MGWR 

Trust,  cooperation 

Self-Efficacy 

MGWR 

Three  indicators  created  from  self-efficacy  items 

Social  Insight 

MGWR 

Three  indicators  created  from  social  insight  items 

Trust 

OC,  SI,  AUCS 

Three  indicators  created  from  trust  items 

Hope/Optimism 

OC 

Three  indicators  created  from  hope/optimism  items 

Extraversion 

SI 

Friendliness,  gregariousness,  assertiveness 

Anxiety 

AUCS 

Three  indicators  created  from  anxiety  items 

Situational 

Flexibility 

AUCS 

Three  indicators  created  from  situational  flexibility 
items 

Note.  The  single-factor  method  was  used  to  create  indicator  variable  whenever  composites  were 
created  from  items.  DMPS  =  Decision  Making/Problem  Solving;  MGWR  =  Maintaining  Good 
Working  Relationships;  OC  =  Organizational  Commitment;  SI  =  Showing  Initiative;  AUCS  = 
Adapting  to  Uncertain  or  Changing  Situations. 

Relationships  Between  Indirect  Performance  Determinants  and  Mediating  Variables 

The  Johnson  (2003)  model  specifies  the  paths  through  which  certain  variables  influence 
performance  but  does  not  address  what  specific  indirect  performance  determinants  influence 
each  mediating  variable.  For  example,  the  model  specifies  that  military  motives  has  a  direct  path 
to  proactive  cognitions  (expectancies  and  goal  commitment),  and  that  personality  influences 
these  motives,  but  the  specific  personality  traits  that  influence  military  motives  are  tangential  to 
the  model.  To  determine  what  paths  involving  indirect  performance  determinants  should  be 
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included  in  the  path  model  beyond  what  are  specified  by  the  general  model,  we  conducted 
multiple  regression  analyses  using  sample  data  to  determine  what  the  significant  predictors  of 
each  variable  are.  Multiple  regression  was  used  to  determine  paths  rather  than  simply  adding  a 
path  wherever  there  was  a  significant  correlation  between  variables  to  avoid  superfluous  paths 
and  keep  the  number  of  parameters  estimated  to  a  minimum. 

In  conducting  the  regression  analyses,  the  independent  variables  were  limited  to  those 
variables  that  are  theoretically  related  to  the  dependent  variable.  For  example,  the  model  predicts 
that  military  motives  is  related  to  self-regulation  through  proactive  cognitions,  so  military 
motives  was  not  included  as  an  independent  variable  when  self-regulation  was  the  dependent 
variable.  Variables  with  significant  regression  coefficients  at/?  <  .10  were  given  direct  paths  to 
the  dependent  variable  in  the  structural  equation  model. 

Based  on  multiple  regression  analyses,  expectancies  and  goal  commitment  were  included 
as  measures  of  proactive  cognitions,  but  self-efficacy  was  not.  Our  original  intention  was  to 
include  a  proactive  cognitions  construct  in  each  model,  with  expectancies,  self-efficacy,  and  goal 
commitment  as  the  indicators.  This  plan  was  abandoned  because  these  variables  exhibited 
differential  relationships  with  other  relevant  variables.  For  example,  expectancies  and  self- 
efficacy  were  both  correlated  with  performance,  but  goal  commitment  was  not.  We  therefore 
included  these  variables  separately  in  regression  analyses  with  self-regulation  or  performance  as 
dependent  variables  and  found  that  expectancies  was  a  significant  predictor  within  all 
performance  dimensions  and  self-efficacy  was  never  significant.  The  pattern  of  correlations 
suggested  that  goal  commitment  may  mediate  the  relationship  between  expectancies  and  self¬ 
regulation  for  some  performance  dimensions.  Goal  commitment  was  a  significant  predictor  of 
performance  as  a  suppressor  variable  for  some  dimensions.  Thus,  expectancies  and  goal 
commitment  were  included  as  separate  variables  and  self-efficacy  was  excluded. 

Models  Tested 

We  used  a  very  structured  approach  to  model  testing  to  ensure  that  we  were  able  to 
identify  the  best-fitting  model  without  capitalizing  on  chance.  We  first  tested  the  measurement 
model,  which  is  the  extent  to  which  the  observed  variables  adequately  represent  the  latent 
constructs.  We  then  tested  two  alternative  a  priori  models,  each  of  which  represented  a 
reasonable  representation  of  the  relationships  between  the  latent  variables.  This  approach  was 
recommended  by  Millsap  (2002).  Model  1  was  based  on  the  original  model,  in  which  habits  had 
direct  paths  to  both  performance  and  self-regulation.  Model  2  was  suggested  by  the  correlations 
observed  in  the  sample  between  habits  and  proactive  cognitions  (see  Table  12).  In  models  in 
which  both  expectancies  and  goal  commitment  were  included,  the  path  from  habits  to  self¬ 
regulation  was  removed  and  a  path  from  habits  to  goal  commitment  was  included.  In  models  in 
which  only  expectancies  were  included,  the  path  from  habits  to  self-regulation  was  removed  and 
a  path  from  habits  to  expectancies  was  included. 

We  then  tested  one  or  more  post  hoc  models  by  making  minor  modifications  to  the  best¬ 
fitting  a  priori  model.  Byrne  (1998)  stated  that  the  post  hoc  addition  or  deletion  of  paths  was  a 
reasonable  way  to  improve  rtiodel  fit  for  structural  equation  models,  as  long  as  any  addition  of 
paths  was  theoretically  meaningful  and  kept  to  a  minimum.  We  followed  this  step  even  if  the  a 
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priori  model  fit  the  data  well  because  we  did  not  want  a  model  that  (a)  included  superfluous 
paths,  and/or  (b)  did  not  adequately  explain  all  relationships  between  indirect  determinants  and 
performance. 

The  first  modifications  we  made  were  eliminating  paths  with  nonsignificant  path 
coefficients.  We  considered  a  path  coefficient  to  be  significant  if  its  p-  value  was  less  than  .  1 0. 
We  used  this  cutoff  so  as  not  to  remove  any  potentially  relevant  paths  and  because  eliminating 
paths  with  coefficients  that  were  significant  at/?  <  .10  always  resulted  in  worse  model  fit. 
Nonsignificant  paths  were  eliminated  one  at  a  time,  starting  with  paths  that  were  not  central  to 
the  model.  After  eliminating  noncentral  paths,  we  eliminated  any  substantive  paths  that  had 
nonsignificant  path  coefficients,  starting  with  the  smallest  coefficient.  This  continued  until  (a)  all 
nonsignificant  paths  were  eliminated,  or  (b)  model  fit  became  significantly  worse  by  eliminating 
paths. 


After  removing  nonsignificant  paths,  we  examined  the  remaining  paths  to  determine  if 
the  model  adequately  explained  relationships  between  indirect  determinants  and  performance.  In 
other  words,  if  an  indirect  determinant  was  correlated  with  performance  but  there  was  no  path 
remaining  through  which  that  variable  could  influence  performance,  the  model  did  not 
adequately  explain  the  data.  We  could  not  examine  modification  indices  because  in  order  for 
Amos  to  use  pairwise  deletion  of  missing  values,  it  was  necessary  to  estimate  means  and 
intercepts.  When  this  is  done,  modification  analyses  cannot  be  computed.  Therefore,  the  addition 
of  paths  was  based  on  conceptual  appropriateness.  A  direct  path  was  added  to  the  most 
conceptually  appropriate  direct  determinant  of  performance.  If  this  path  was  not  significant,  it 
was  deleted  and  a  path  added  to  another  direct  determinant.  If  no  paths  through  direct 
determinants  were  significant,  we  tried  a  direct  path  from  the  variable  to  performance.  If  that 
path  was  not  significant,  we  concluded  that  the  correlation  between  that  variable  and 
performance  was  due  to  relationships  with  other  variables  rather  than  a  causal  relationship. 

Testing  Model  Fit 

Overall  Fit  Statistics.  A  number  of  goodness  of  fit  statistics  are  available  to  evaluate  the 
fit  of  each  model.  We  used  four  that  we  feel  are  most  informative.  Chi-square  is  a  measure  of  the 
distance  between  the  sample  covariance  matrix  and  the  covariance  matrix  suggested  by  the 
model  (Joreskog  &  Sorbom,  1993).  This  index  increases  with  sample  size  and  is  based  on  the 
assumption  that  the  model  holds  exactly  in  the  population.  The  assumption  that  the  model  holds 
exactly  in  the  population  may  be  unreasonable,  so  this  is  taken  into  account  by  Steiger’s  (1990) 
Root  Mean  Square  Error  of  Approximation  (RMSEA).  RMSEA  is  really  a  test  of  whether  the 
model  fits  the  data  reasonably  well.  According  to  Browne  and  Cudeck  (1993),  a  RMSEA  of  .05 
or  less  is  an  indication  of  close  fit,  and  values  as  high  as  .08  are  reasonable.  More  recently,  Hu 
and  Bentler  (1998,  1999)  recommended  an  RMSEA  of  .06  or  less  as  an  indication  of  close  fit. 

Bentler’s  (1990)  Comparative  Fit  Index  (CFI)  compares  the  chi-square  for  the  target 
model  to  the  chi-square  for  a  baseline  (usually  the  null)  model.  CFI  is  relatively  insensitive  to 
sample  size  and  has  been  shown  to  have  desirable  properties  in  terms  of  being  sensitive  to  model 
misspecification  (Lance  &  Vandenberg,  2002).  Hu  and  Bentler  (1998, 1999)  recommended  a 
CFI  of  at  least  .95  as  an  indication  of  close  model  fit. 
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Finally,  the  Expected  Cross-Validation  Index  (ECVI)  (Browne  &  Cudeck,  1989)  is  a 
measure  of  the  discrepancy  between  the  fitted  covariance  matrix  in  the  data  analyzed  and  the 
expected  covariance  matrix  that  would  be  obtained  in  another  sample  of  equal  size  (Joreskog  & 
Sorbom,  1993).  It  has  the  property  of  decreasing  only  if  additional  paths  substantially  reduce  the 
estimate  of  discrepancy,  and  increasing  if  superfluous  paths  are  hypothesized.  The  best-fitting 
model  among  a  set  of  alternatives  is  then  the  model  at  which  ECVI  is  at  its  minimum. 

Path  Model  Fit.  In  structural  equation  modeling,  the  structural  model  is  a  composite  of 
the  measurement  model  and  the  path  model.  The  measurement  model  represents  a  set  of 
observed  variables  that  serve  as  indicators  of  a  set  of  latent  variables.  The  path  model  describes 
(usually  causal)  relationships  between  the  latent  variables.  Because  the  primary  objective  of 
structural  equation  modeling  is  to  test  a  specified  path  model,  McDonald  and  Ho  (2002) 
recommended  separating  the  fit  of  the  path  model  from  the  fit  of  the  measurement  model.  They 
pointed  out  that  the  fit  of  the  structural  model  can  appear  satisfactory  because  of  a  well-fitting 
measurement  model,  even  when  the  paths  specified  in  the  path  model  are  not  correctly  specified. 
Conversely,  the  path  model  may  be  correctly  specified  but  a  misspecified  measurement  model 
could  make  the  entire  structural  model  appear  to  be  misspecified. 

Using  a  sequential  testing  procedure  suggested  by  Anderson  and  Gerbing  (1988), 
McDonald  and  Ho  (2002)  demonstrated  how  chi-square  and  RMSEA  can  be  calculated  for  the 
path  model  independent  of  the  measurement  model.  Because  chi-squares  and  degrees  of  freedom 
are  additive  for  nested  models,  the  chi-square  for  the  path  model  is  obtained  by  subtracting  the 
chi-square  for  the  measurement  model  from  the  chi-square  for  the  structural  model  (and  similarly 
for  degrees  of  freedom).  RMSEA,  CFI,  and  ECVI  can  be  computed  based  on  these  differences  in 
chi-squares  and  degrees  of  freedom,  as  well  as  the  number  of  distinct  sample  moments  and  the 
noncentrality  parameter,  which  are  provided  in  the  Amos  output. 

We  tested  the  path  model  independent  of  the  measurement  model  for  each  of  the  five 
performance  dimensions.  In  each  case,  the  chi-square  for  the  final  model  was  nonsignificant, 
indicating  that  the  hypothesis  that  the  specified  path  model  fits  the  data  could  not  be  rejected.  In 
other  words,  the  path  model  describing  relationships  between  the  latent  variables  provided  a 
good  fit  to  the  data  independent  of  the  measurement  model  describing  relationships  between  the 
observed  variables  and  the  latent  variables. 

Evaluation  of  Structural  Equation  Models 
Decision  Making/Problem  Solving 

In  addition  to  cognitive  ability  and  experience,  we  included  conscientiousness,  emotional 
stability,  and  intellectance  as  other  indirect  performance  determinants  in  this  model.  Sample 
correlations  between  the  variables  included  in  this  model  are  presented  in  Table  16.  Figure  2 
shows  how  the  path  model  depicted  in  Figure  1  was  translated  into  a  testable  model  for  the 
dimension  of  decision  making/problem  solving.  For  simplicity,  the  figure  excludes  measured 
variables,  error  variances,  and  correlations  between  exogenous  variables.  This  figure  depicts 
Model  1 .  To  visualize  Model  2,  the  path  from  work  habits  to  self-regulation  would  be  removed 
and  replaced  with  a  path  from  work  habits  to  goal  commitment. 
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Fit  statistics  for  all  models  tested  (i.e.,  measurement  model,  a  priori  models,  post  hoc 
models),  including  those  for  the  path  models  independent  of  the  measurement  model,  are 
presented  in  Table  1 7.  When  adding  the  path  from  work  habits  to  goal  commitment  in  Model  2, 
we  also  removed  the  paths  from  intellectance  to  goal  commitment  and  from  cognitive  ability  to 
goal  commitment,  because  the  regression  analyses  indicated  that  these  variables  did  not 
contribute  significantly  beyond  work  habits.  Model  2  fit  significantly  better  than  Model  1 . 
Examining  the  fit  statistics  independent  of  the  measurement  model  shows  that  the  hypothesis  that 
Model  2  does  not  fit  the  data  could  not  be  rejected  (p  =  .104).  All  hypothesized  direct  paths  to 
performance  were  significant,  except  work  habits  was  a  suppressor  variable. 
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Figure  2.  Operationalization  of  Johnson  (2003)  model  for  Decision  Making/Problem  Solving. 


Table  17 

Goodness  of  Fit  Statistics  for  Decision  Making/Problem  Solving  Models  Tested 


Model 

df 

X2 

/?-value 

RMSEA 

ECVI 

CFI 

Measurement  model 

674 

1,066.1 

.000 

.061 

9.34 

.87 

A  priori  models: 
Model  1 

707 

1,124.0 

.000 

.062 

9.29 

.86 

Model  2 

709 

1,112.0 

.000 

.061 

9.18 

.87 

Post  hoc  models: 

Model  3 

707 

1,097.2 

.000 

.060 

9.11 

.87 

Model  4 

711 

1,101.6 

.000 

.060 

9.09 

.87 

Path  Model  Independent  of  Measurement  Model 

Model 

df 

y.2  -  -  - 

p-value 

RMSEA 

ECVI 

CFI 

A  priori  models: 
Model  1 

33 

57.9 

.005 

.070 

11.12 

.94 

Model  2 

35 

45.9 

.104 

.045 

11.01 

.97 

Post  hoc  models: 

Model  3 

33 

31.1 

.562 

_ * 

10.94 

1.00 

Model  4 

37 

35.4 

.543 

_ * 

10.92 

1.00 

Note.  A  priori  models  are  described  in  text.  N=  155.  *  RMSEA  could  not  be  computed  because 
of  a  negative  square  root. 

The  problem  with  Model  2  was  that  the  path  from  goal  commitment  to  self-regulation 
was  not  significant,  which  meant  that  any  path  going  through  proactive  cognitions  did  not  reach 
performance  that  way.  The  regression  analyses  suggested  that  expectancies  and  goal 
commitment  were  directly  related  to  performance,  with  goal  commitment  acting  as  a  suppressor 
variable.  In  Model  3,  we  applied  the  paths  suggested  by  the  regression  results  for  self-regulation 
and  performance.  This  meant  (a)  deleting  the  path  from  goal  commitment  to  self-regulation,  (b) 
adding  the  path  from  experience  to  self-regulation,  (c)  adding  paths  from  expectancies  and  goal 
commitment  to  performance,  and  (d)  deleting  the  path  from  work  habits  to  performance.  Goal 
commitment  took  over  the  suppressor  variable  role  played  by  work  habits  in  the  previous  model. 
Model  3  fit  significantly  better  than  Model  2,  and  all  paths  to  performance  were  significant. 
Model  4  removed  four  nonsignificant  paths  that  were  not  central  to  the  model.  The  final  model  is 
displayed  in  Figure  3,  including  standardized  path  coefficients. 
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Knowledge 


Figure  3.  Final  performance  prediction  model  for  Decision  Making/Problem  Solving. 


There  are  two  meaningful  differences  between  the  final  model  presented  in  Figure  3  and 
the  original  model  proposed  in  Figure  2.  First,  both  proactive  cognitions  and  self-regulation  have 
direct  effects  on  performance,  rather  than  proactive  cognitions  having  their  effect  through  self¬ 
regulation.  This  makes  sense,  because  many  intentions  may  not  require  active  self-regulatory 
strategies  to  enact  them.  Thus,  self-regulation  is  a  direct  determinant  but  not  necessarily  at  all 
times.  The  second  difference  is  the  relationships  involving  work  habits.  Rather  than  influencing 
both  self-regulation  and  performance  directly,  work  habits  influence  performance  through  goal 
commitment.  Those  whose  work  habits  facilitate  the  performance  of  behavior  relevant  to 
decision  making/problem  solving  tend  to  be  more  committed  to  performing  those  behaviors. 

Goal  commitment,  however,  does  not  lead  directly  to  performance.  Removing  the  variance  in 
expectancies  that  is  related  to  goal  commitment  but  unrelated  to  performance  strengthens  the 
relationship  between  expectancies  and  performance. 

Maintaining  Good  Working  Relationships 

For  the  maintaining  good  working  relationships  model,  we  included  agreeableness, 
generalized  self-efficacy,  and  social  insight,  with  cognitive  ability  and  experience  as  indirect 
performance  determinants.  Preliminary  analyses  indicated  that  goal  commitment  did  not 
contribute  meaningfully  to  the  model,  so  expectancies  was  the  only  proactive  cognition  variable 
included.  Sample  correlations  between  the  variables  included  in  this  model  are  presented  in 
Table  18.  Fit  statistics  for  all  models  tested  are  presented  in  Table  19. 

In  Model  2,  we  deleted  the  path  from  work  habits  to  self-regulation  and  added  a  path 
from  work  habits  to  expectancies.  Model  1  and  Model  2  fit  the  data  equally  well,  but  knowledge 
was  the  only  direct  determinant  of  performance  that  had  a  significant  path  coefficient. 

In  Model  3,  we  removed  two  nonessential  nonsignificant  paths  and  the  path  from  work 
habits  to  performance.  Skill  and  self-regulation  still  had  nonsignificant  paths  to  performance,  so 
those  paths  were  removed  in  Model  4.  We  also  removed  a  nonsignificant  path  from  expectancies 
to  self-regulation  and  added  a  path  from  experience  to  self-regulation,  which  was  suggested  by 
the  regression  analyses.  That  model  fit  significantly  better  than  Model  3,  but  it  left  several 
variables  with  no  path  to  performance.  The  most  logical  next  step  was  a  direct  path  to 
performance  from  expectancies,  but  this  path  was  not  significant.  The  regression  analyses 
suggested  that  military  motives  was  directly  related  to  performance,  so  this  path  was  added  in 
Model  5.  This  path  was  significant  and  resulted  in  significantly  better  model  fit.  This  still  left 
experience  and  expectancies  as  variables  that  are  correlated  with  performance  but  do  not  have  a 
path  to  performance.  We  tried  adding  direct  paths  from  both  of  these  variables  to  performance 
but  neither  path  was  significant  and  both  detracted  from  model  fit.  These  variables’  relationship 
with  performance  may  be  a  result  of  being  correlated  with  other  variables  that  are  determinants 
of  performance.  Figure  4  presents  the  path  diagram  for  Model  5,  including  standardized  path 
coefficients. 
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Table  18 

Sample  Correlations  Between  Variables  Included  in  the  Maintaining  Good  Working  Relationships  Structural  Model 
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.Table  19 

Goodness  of  Fit  Statistics  for  Maintaining  Good  Working  Relationships  Models  Tested 


Model 

df 

x2 

/j-value 

RMSEA 

ECVI 

CFI 

Measurement  model 

A  priori  models: 

540 

797.4 

.000 

.055 

7.28 

.90 

Model  1 

565 

835.0 

.000 

.056 

7.20 

.90 

Model  2 

565 

834.9 

.000 

.056 

7.20 

.90 

Post  hoc  models: 

Model  3 

568 

835.1 

.000 

.055 

7.16 

.90 

Model  4 

570 

829.7 

.000 

.054 

7.10 

.90 

Model  5 

569 

824.0 

.000 

.054 

7.08 

.90 

Path  Model  Independent  of  Measurement  Model 

Model 

df 

x2 

/>-value 

RMSEA 

ECVI 

CFI 

A  priori  models: 
Model  1 

25 

37.6 

.051 

.057 

9.04 

.95 

Model  2 

25 

37.4 

.052 

.057 

9.04 

.95 

Post  hoc  models: 

Model  3 

28 

37.7 

.105 

.047 

9.00 

.96 

Model  4 

30 

32.3 

.356 

.022 

8.94 

.99 

Model  5 

29 

26.5 

.596 

_ * 

8.91 

1.00 

Note.  A  priori  models  are  described  in  text.  N=  155.  *RMSEA  could  not  be  computed  because 
of  a  negative  square  root. 


56 


Figure  4.  Final  performance  prediction  model  for  Maintaining  Good  Working  Relationships. 


Organizational  Commitment 


For  the  organizational  commitment  model,  we  included  trust,  hope/optimism,  and 
conscientiousness  with  cognitive  ability  and  experience  as  indirect  performance  determinants. 
Preliminary  analyses  indicated  that  goal  commitment  did  not  contribute  meaningfully  to  the 
model  so  expectancies  were  the  only  proactive  cognition  variable  included.  In  addition,  the 
organizational  commitment  PBR  was  uncorrelated  with  all  other  variables,  so  skill  was  not 
included  as  a  construct  in  the  model.  Sample  correlations  between  the  variables  included  in  this 
model  are  presented  in  Table  20.  Fit  statistics  for  all  models  tested  are  presented  in  Table  21 . 

This  model  differed  from  Model  1  for  the  other  performance  dimensions  in  that  military 
motives  was  a  direct  determinant  of  work  habits.  We  believed  this  path  made  sense  for  the 
organizational  commitment  performance  dimension  because  habitual  behaviors  relevant  to 
organizational  commitment  would  depend  on  the  level  of  commitment  that  an  individual 
possesses.  This  would  not  be  the  case  for  the  other  performance  dimensions. 

In  Model  2,  we  deleted  the  path  from  work  habits  to  self-regulation  and  added  a  path 
from  work  habits  to  expectancies.  We  also  deleted  the  path  from  military  motives  to 
expectancies  because  it  was  not  significant  in  Model  1  and  it  was  not  supported  by  the  regression 
analyses.  Model  1  and  Model  2  fit  the  data  about  equally  well,  but  self-regulation  was  the  only 
direct  determinant  of  performance  that  had  a  significant  path  coefficient. 

In  Model  3,  we  removed  two  nonessential  nonsignificant  paths  and  the  paths  from  work 
habits  and  knowledge  to  performance.  In  Model  4,  the  nonsignificant  path  from  expectancies  to 
self-regulation  was  removed  and  a  direct  path  from  expectancies  to  performance  was  added.  This 
path  was  not  significant,  so  we  deleted  it  and  added  a  direct  path  from  military  motives  to 
performance,  which  was  suggested  by  the  regression  analyses.  That  model  fit  significantly  better 
than  Model  3,  but  experience  had  no  path  to  performance  despite  the  high  correlation  between 
the  two  variables.  Paths  to  military  motives  and  self-regulation  were  not  significant,  so  Model  5 
added  a  direct  path  from  experience  to  performance.  This  path  was  significant  and  resulted  in 
significantly  better  model  fit.  This  model  provides  a  path  to  performance  from  every  variable 
that  is  correlated  with  performance,  but  the  hypothesis  that  this  model  adequately  fits  the  data 
was  still  rejected  (p  =  .03).  This  meant  that  there  should  be  at  least  one  path  from  one  construct 
to  another  that  is  not  central  to  the  model  but  needs  to  be  specified  to  make  the  model  fit  well. 
Examining  the  sample  correlations,  it  appeared  that  the  path  that  made  the  most  sense  was  from 
military  motives  to  interests.  The  level  of  interest  an  individual  has  in  performing  behaviors 
relevant  to  organizational  commitment  should  depend  on  that  individual’s  level  of  job 
satisfaction  and  affective  commitment.  This  path  was  added  in  Model  6,  resulting  in  a  significant 
increase  in  model  fit.  The  hypothesis  that  Model  6  represents  the  data  well  could  not  be  rejected 
( p  =  .29).  Figure  5  presents  the  path  diagram  for  Model  6,  including  standardized  path 
coefficients.  Two  path  coefficients  are  not  significant,  including  the  path  from  self-regulation  to 
performance,  but  removing  them  significantly  reduces  model  fit.  This  suggests  that  these  paths 
are  important  for  explaining  the  relationships  between  variables  and  they  simply  require  an 
increase  in  sample  size  to  reach  an  acceptable  level  of  significance. 
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Table  21 

Goodness  of  Fit  Statistics  for  Organizational  Commitment  Models  Tested 


Model 

df 

x2 

p-value 

RMSEA 

ECVI 

CFI 

Measurement  model 

511 

781.8 

.000 

.058 

7.08 

.91 

A  priori  models: 
Model  1 

532 

833.4 

.000 

.060 

7.14 

.90 

Model  2 

533 

832.8 

.000 

.060 

7.12 

.90 

Post  hoc  models: 

Model  3 

537 

835.2 

.000 

.060 

7.09 

.90 

Model  4 

537 

827.2 

.000 

.059 

7.03 

.90 

Model  5 

536 

822.3 

.000 

.059 

7.02 

.90 

Model  6 

536 

810.1 

.000 

.057 

6.94 

.90 

Path  Model  Independent  of  Measurement  Model 

Model 

df 

x2 

/j-value 

RMSEA 

ECVI 

CFI 

A  priori  models: 
Model  1 

21 

51.6 

.000 

.097 

8.70 

.89 

Model  2 

22 

51.0 

.000 

.092 

8.68 

.89 

Post  hoc  models: 

Model  3 

26 

53.4 

.001 

.082 

8.65 

.90 

Model  4 

26 

45.3 

.011 

.069 

8.59 

.93 

Model  5 

25 

40.5 

.026 

.063 

8.57 

.94 

Model  6 

25 

28.3 

.294 

.029 

8.50 

.99 

Note.  A  priori  models  are  described  in  text.  N=  155. 
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Figure  5.  Final  performance  prediction  model  for  Organizational  Commitment. 


Showing  Initiative 

For  the  showing  initiative  model,  we  included  trust,  extraversion,  intellectance,  and 
conscientiousness  with  cognitive  ability  and  experience  as  indirect  performance  determinants. 
Conscientiousness  was  not  significantly  correlated  with  performance,  but  we  included  it  in  the 
model  because  it  is  highly  conceptually  related  to  showing  initiative.  Preliminary  analyses 
indicated  that  goal  commitment  did  not  contribute  meaningfully  to  the  model  so  expectancies 
was  the  only  proactive  cognition  variable  included.  Sample  correlations  between  the  variables 
included  in  this  model  are  presented  in  Table  22.  Fit  statistics  for  all  models  tested  are  presented 
in  Table  23. 

In  Model  2,  we  removed  the  path  from  interests  to  expectancies  when  the  path  from  work 
habits  to  expectancies  was  added  because  it  was  nonsignificant  in  Model  1  and  the  path  from 
interests  to  expectancies  would  then  go  through  work  habits.  Work  habits  was  unrelated  to  both 
self-regulation  and  expectancies,  so  Model  1  and  Model  2  fit  equally  well.  In  Model  3,  we 
removed  the  nonsignificant  paths,  including  the  paths  from  knowledge  and  work  habits  to 
performance  and  the  path  from  military  motives  to  expectancies.  This  left  only  skill  and  self¬ 
regulation  as  determinants  of  performance,  and  military  motives  did  not  have  a  path  to 
performance  but  was  highly  correlated  with  performance.  In  Model  4,  we  added  a  direct  path 
from  military  motives  to  performance.  This  significantly  improved  the  fit  of  the  model.  It  also 
made  the  path  from  self-regulation  to  performance  nonsignificant,  so  this  path  was  removed. 

Intellectance  was  the  only  variable  correlated  with  performance  that  did  not  have  a  path 
to  performance.  The  direct  path  was  not  significant.  The  path  to  military  motives  was  not  quite 
significant  (p  =  .12),  but  it  did  reduce  ECVI  and  it  explains  the  relationship  between  intellectance 
and  performance,  so  Model  5  includes  this  path.  Figure  6  presents  the  path  diagram  for  Model  5, 
including  standardized  path  coefficients. 

Adapting  to  Uncertain  or  Changing  Situations 

For  the  adapting  to  uncertain  or  changing  situations  model,  we  included  trust,  anxiety, 
intellectance,  and  situational  flexibility  with  cognitive  ability  and  experience  as  indirect 
performance  determinants.  Situational  flexibility  was  uncorrelated  with  performance,  but  it  was 
included  because  it  is  so  conceptually  relevant  to  adaptive  performance.  Goal  commitment  was 
included  in  this  model  because  preliminary  analyses  suggested  that  it  was  a  likely  suppressor 
variable.  Sample  correlations  between  the  variables  included  in  this  model  are  presented  in  Table 
24.  Fit  statistics  for  all  models  tested  are  presented  in  Table  25. 

Model  2  fit  significantly  better  than  Model  1,  but  the  only  significant  path  to  performance 
was  from  knowledge.  In  Model  3,  we  removed  the  nonsignificant  paths  from  self-regulation, 
work  habits,  and  skill  to  performance,  as  well  as  two  nonsignificant  paths  that  were  not  central  to 
the  model.  Because  the  path  from  self-regulation  to  performance  was  removed,  we  added  direct 
paths  to  performance  from  expectancies  and  goal  commitment  in  Model  4.  Goal  commitment 
was  a  suppressor  variable  that  strengthened  the  relationship  between  expectancies  and 
performance. 
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Table  22 

Sample  Correlations  Between  Variables  Included  in  the  Showing  Initiative  Structural  Model 

Variable  1  2  3  4  "  5  6  7 

1 .  Performance  1 .00 

2.  Skill  .18*  1.00 
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Table  23 

Goodness  of  Fit  Statistics  for  Showing  Initiative  Models  Tested 


Model 

df 

x2 

/7-value 

RMSEA 

ECVI 

CFI 

Measurement  model 

715 

1,065.6 

.000 

.056 

9.35 

.90 

A  priori  models: 
Model  1 

744 

1,103.8 

.000 

.056 

9.22 

.89 

Model  2 

745 

1,104.0 

.000 

.056 

9.21 

.89 

Post  hoc  models: 

Model  3 

752 

1,109.4 

.000 

.055 

9.15 

.89 

Model  4 

752 

1,106.3 

.000 

.055 

9.13 

.90 

Model  5 

751 

1,104.3 

.000 

.055 

9.13 

.90 

Path  Model  Independent  of  Measurement  Model 

Model 

df 

x2 

p- value 

RMSEA 

ECVI 

CFI 

A  priori  models: 
Model  1 

29 

38.2 

.118 

.045 

11.59 

.97 

Model  2 

30 

38.4 

.141 

.042 

11.57 

.98 

Post  hoc  models: 

Model  3 

37 

43.7 

.207 

.034 

11.52 

.98 

Model  4 

37 

40.7 

.312 

.025 

11.50 

.99 

Model  5 

36 

38.6 

.352 

.022 

11.50 

.99 

Note.  A  priori  models  are  described  in  text.  N=  155. 
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Figure  6.  Final  performance  prediction  model  for  Showing  Initiative. 


Every  variable  that  is  correlated  with  performance  has  a  path  to  performance  in  Model  4, 
but  the  fit  statistics  independent  of  the  measurement  model  indicate  an  inadequate  level  of  fit. 
Trust  is  more  highly  correlated  with  performance  than  would  be  suggested  by  its  distance  from 
the  criterion  (trust  ->  military  motives  ->  expectancies  ->  perform).  It  is  more  highly  correlated 
with  performance  than  are  military  motives  and  expectancies,  which  means  there  must  be 
another  path  to  performance  through  which  trust  operates.  Trust  is  not  correlated  with  knowledge 
but  it  is  correlated  with  goal  commitment,  so  a  path  was  added  from  trust  to  goal  commitment. 
This  path  was  not  significant,  so  we  added  a  direct  path  to  performance  in  Model  5.  The  path  to 
performance  was  significant  and  made  a  significant  increase  in  the  fit  of  the  model,  but  the  fit 
statistics  independent  of  the  measurement  model  still  did  not  indicate  close  fit  {p  =  .001 1, 
RMSEA  =  .082,  CFI  =  .90). 

As  with  the  showing  initiative  model,  there  must  have  been  one  or  more  missing  paths 
that  are  not  central  to  the  model.  Examining  the  correlations  in  Table  24,  we  noticed  that 
knowledge  and  goal  commitment  were  correlated  .40  and  considered  the  likelihood  that 
knowledge  is  causally  related  to  goal  commitment.  In  a  review  of  the  antecedents  of  goal  setting, 
Wofford,  Goodwin  and  Premack  (1992)  found  that  the  major  determinants  of  goal  level  were 
past  performance  and  ability.  It  follows  that  those  with  greater  knowledge  relevant  to  adapting 
would  be  more  likely  to  set  goals  relevant  to  adaptive  performance  and  be  committed  to  them. 
Model  6  added  a  path  from  knowledge  to  goal  commitment,  creating  a  significant  improvement 
in  model  fit.  Nevertheless,  the  hypothesis  that  the  model  fits  the  data  was  still  rejected  (p  <  .05). 

As  with  the  organizational  commitment  model,  we  believed  that  military  motives  may  be 
a  determinant  of  interests.  Job  satisfaction,  military  values,  affective  commitment,  and  interests 
are  all  potential  motives  and  are  intercorrelated,  and  it  makes  sense  that  the  more  general  military 
motives  would  lead  to  the  more  job-specific  interest  in  participating  in  situations  that  involve 
adapting.  We  added  this  path  in  Model  7  and  it  resulted  in  a  significant  improvement  in  model  fit 
and  the  hypothesis  that  the  model  fits  the  data  could  not  be  rejected  (p  =  .27).  Figure  7  presents 
the  path  diagram  for  Model  7,  including  standardized  path  coefficients. 
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Table  24 

Sample  Correlations  Between  Variables  Included  in  the  Adapting  to  Uncertain  or  Changing  Situations  Structural  Model 

Variable  123  4  5  6  7  8  9  10  11  ~~12~ 

1 .  Performance  1 .00 

2.  Skill  .08  1.00 
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Table  25 

Goodness  of  Fit  Statistics  for  Adapting  to  Uncertain  or  Changing  Situations  Models  Tested 


Model 

df 

x2 

p-value 

RMSEA 

ECVI 

CFI 

Measurement  model 

794 

1,075.2 

.000 

.048 

9.51 

.93 

A  priori  models: 
Model  1 

819 

1,146.9 

.000 

.051 

9.66 

.92 

Model  2 

819 

1,138.7 

.000 

.050 

9.60 

.92 

Post  hoc  models: 

Model  3 

824 

1,143.7 

.000 

.050 

9.57 

.92 

Model  4 

822 

1,135.9 

.000 

.050 

9.54 

.92 

Model  5 

821 

1,130.5 

.000 

.049 

9.52 

.93 

Model  6 

821 

1,115.6 

.000 

.048 

9.43 

.93 

Model  7 

820 

1,105.1 

.000 

.047 

9.37 

.93 

Path  Model  Independent  of  Measurement  Model 

Model 

df 

y.2 

p-value 

RMSEA 

ECVI 

CFI 

A  priori  models: 

Model  1 

25 

71.7 

.000 

.110 

12.99 

.83 

Model  2 

25 

63.5 

.000 

.100 

12.93 

.86 

Post  hoc  models: 

Model  3 

30 

68.5 

.000 

.091 

12.90 

.86 

Model  4 

28 

60.6 

.000 

.087 

12.87 

.88 

Model  5 

27 

55.3 

.001 

.082 

12.85 

.90 

Model  6 

27 

40.4 

.047 

.057 

12.76 

.95 

Model  7 

26 

29.9 

.273 

.031 

12.70 

.99 

Note.  A  priori  models  are  described  in  text.  N=  155. 
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Figure  7.  Final  performance  prediction  model  for  Adapting  to  Uncertain  or  Changing  Situations. 


Discussion 


In  this  research,  we  used  a  comprehensive  general  model  of  individual  job  performance 
to  examine  the  process  by  which  individual  differences  influence  performance  on  five  different 
dimensions.  This  model  included  four  direct  determinants  of  performance  (knowledge,  skill, 
motivation,  and  work  habits),  five  components  of  motivation  (motives,  expectancies,  self- 
efficacy,  goal  commitment,  and  self-regulation),  and  a  host  of  potential  indirect  performance 
determinants  from  the  personality,  ability,  experience,  action  control,  and  social  intelligence 
domains.  We  identified  five  performance  constructs  and  demonstrated  that  they  were  all 
important  aspects  of  the  ROTC  cadet/midshipman  performance  domain.  We  then  tested  the 
performance  model  within  each  of  these  performance  constructs. 

The  level  of  support  for  the  model  depended  on  the  performance  construct  to  which  it 
was  applied.  In  the  following  sections,  we  discuss  the  results  for  each  performance  construct, 
how  the  best-fitting  model  differed  from  the  original  model,  and  the  implications  of  the  results. 

Key  Findings 


Decision  Making/Problem  Solving 

Decision  making/problem  solving  was  the  only  dimension  of  task  performance  that  was 
studied.  Previous  tests  of  performance  models  have  been  limited  to  task  performance  and  results 
have  consistently  shown  that  the  relationship  between  cognitive  ability  and  performance  is 
mediated  by  job  knowledge  and  skill  (e.g.,  Borman  et  al.,  1991;  Hunter,  1983;  Lance  &  Bennett, 
2000;  Schmidt  et  ah,  1986).  Our  results  were  consistent  with  previous  research,  except  cognitive 
ability  was  not  a  significant  predictor  of  performance.  Rather,  personality  was  the  dominant 
predictor  of  performance,  but  the  mediating  effect  of  knowledge  and  skill  also  was  found.  In 
addition  to  knowledge  and  skill,  four  components  of  motivation  also  were  also  found  to  mediate 
the  relationship  between  personality  and  performance.  This  mediating  function  of  motivation  has 
been  postulated  in  models  by  Campbell  et  ah  (1993)  and  Motowidlo  et  ah  (1997),  and 
demonstrated  in  studies  by  Barrick,  Mount,  and  Strauss  (1993),  Gellatly  (1996),  and  Bamck, 
Stewart,  and  Piotrowski  (2002).  However,  this  is  the  first  research  to  show  the  mediating  effects 
of  knowledge,  skill,  and  motivation  simultaneously. 

For  the  decision  making/problem  solving  model,  there  were  four  direct  determinants  of 
performance:  (a)  knowledge,  (b)  skill,  (c)  expectancies,  and  (d)  self-regulation.  Goal 
commitment  also  had  a  direct  path,  but  as  a  suppressor  variable  that  strengthened  the  path  from 
expectancies.  This  was  contrary  to  the  Johnson  (2003)  model,  which  posits  that  all  components 
of  proactive  cognitions  should  be  positively  related  to  performance.  Further  research  is  necessary 
to  understand  this  complex  relationship  between  expectancy,  goal  commitment,  and  decision 
making/problem  solving  performance. 

Also  contrary  to  the  Johnson  (2003)  model,  work  habits  were  not  a  direct  determinant  of 
decision  making/problem  solving  performance.  Rather,  work  habits  had  an  indirect  influence 
through  goal  commitment.  Work  habits  are  only  expected  to  bypass  the  motivation  component 
and  be  a  direct  determinant  of  performance  when  the  relevant  behaviors  occur  automatically.  It  is 
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very  likely  that  the  kinds  of  behaviors  assessed  in  this  project  do  not  occur  automatically,  so  the 
lack  of  a  direct  path  does  not  necessarily  invalidate  that  portion  of  the  model.  Work  habits  may 
be  more  likely  to  have  a  direct  effect  when  the  measurement  of  performance  is  on  simpler  or 
more  specific  behaviors  in  specific  situations. 

Work  habits  also  were  expected  to  be  a  direct  determinant  of  self-regulation,  because 
individuals  whose  work  habits  interfere  with  their  ability  to  behave  in  accordance  with  good 
performance  would  be  more  likely  to  require  self-regulatory  strategies  to  maintain  goal-directed 
performance.  Although  work  habits  were  correlated  with  self-regulation,  it  was  not  a  significant 
predictor  in  the  presence  of  other  variables. 

The  relationship  between  the  motives  and  proactive  cognitions  components  of  motivation 
was  consistent  with  the  Johnson  (2003)  model.  The  relationship  between  motives  and 
performance  was  expected  to  be  mediated  by  proactive  cognitions.  In  the  decision 
making/problem  solving  model,  general  military  motives  had  a  direct  path  to  expectancies  and 
interests  had  a  direct  path  to  goal  commitment.  Contrary  to  the  Johnson  model,  the  relationship 
between  proactive  cognitions  and  performance  was  not  mediated  by  self-regulation.  Instead,  self¬ 
regulation  made  an  independent  contribution  to  performance  in  concert  with  the  contribution 
made  by  proactive  cognitions.  This  is  not  too  surprising,  because  many  behaviors  do  not  require 
much  self-regulation  to  enact  them,  especially  if  the  environment  facilitates  goal  maintenance. 
Also,  the  strength  of  one’s  motivation  to  perform  a  behavior  does  not  necessarily  lead  to  greater 
success  in  self-regulating  (Heckhausen  &  Kuhl,  1985). 

The  best-fitting  model  for  decision  making/problem  solving  is  generally  very  consistent 
with  the  Johnson  (2003)  model.  The  different  components  of  motivation  are  distinct  and 
influence  performance  in  the  order  suggested  by  the  model.  All  personality  and  experience 
variables  influence  performance  indirectly  through  knowledge,  skill,  or  motivation.  Despite  the 
complexity  of  the  model,  fit  statistics  for  the  path  model  independent  of  the  measurement  model 
show  extremely  good  fit  of  the  model  to  the  data. 

Maintaining  Good  Working  Relationships 

The  best-fitting  model  for  maintaining  good  working  relationships  differed  quite  a  bit 
from  the  Johnson  (2003)  model  and  the  decision  making/problem  solving  model.  Of  the 
hypothesized  direct  determinants  of  performance,  only  knowledge  had  a  significant  path 
coefficient.  Skill,  expectancies,  and  self-regulation  were  not  direct  determinants  of  performance. 
The  other  direct  determinant  was  a  component  of  motivation,  but  it  was  general  military  motives 
rather  than  a  proactive  cognitions  component.  This  suggests  that  attitudes  such  as  job  satisfaction 
and  affective  commitment  can  lead  directly  to  cooperative,  social  behaviors  that  support  others 
without  any  cognition  directed  at  evaluating  possible  consequences  or  setting  goals. 

Maintaining  good  working  relationships  is  an  aspect  of  social  performance,  and  this 
model  is  consistent  with  the  results  of  Schneider  and  Johnson’s  (2005)  test  of  a  model  of  socially 
competent  job  performance.  For  three  of  the  five  social  performance  dimensions  they  studied, 
social  knowledge  mediated  the  relationship  between  indirect  performance  determinants  (social 
insight,  personality,  and  cognitive  ability)  and  performance.  Motivation  had  a  significant  positive 
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path  to  performance  in  only  one  of  the  five  models.  Schneider  and  Johnson’s  measure  of 
motivation  focused  primarily  on  the  importance  respondents  placed  on  performing  behaviors 
relevant  to  each  performance  dimension,  placing  it  in  the  proactive  cognitions  category  of 
motivation.  In  our  model,  we  found  that  knowledge  relevant  to  maintaining  good  working 
relationships  mediates  the  relationship  between  indirect  performance  determinants 
(agreeableness,  cognitive  ability)  and  performance.  Proactive  cognitions  (i.e.,  expectancies)  did 
not  have  a  significant  path  to  performance.  We  found  that  general  motives  (job  satisfaction, 
military  values,  and  affective  commitment)  were  directly  related  to  performance,  but  Schneider 
and  Johnson  did  not  include  any  of  these  types  of  variables  in  their  study.  Thus,  limiting  the 
constructs  to  those  measured  in  both  studies,  the  models  are  very  consistent. 

This  model  does  a  good  job  of  demonstrating  why  agreeableness  is  more  highly 
correlated  with  performance  than  are  self-efficacy  or  social  insight.  Maintaining  good  working 
relationships  performance  is  a  function  of  knowledge  and  general  military  motives.  Self-efficacy 
and  social  insight  are  determinants  of  military  motives,  but  agreeableness  is  a  determinant  of 
both  military  motives  and  knowledge.  Thus,  agreeableness  has  a  higher  correlation  with 
performance,  as  would  be  predicted  by  the  Johnson  (2003)  model. 

Organizational  Commitment 

The  model  for  organizational  commitment  is  very  different  from  the  Johnson  (2003) 
model.  Although  self-regulation  is  a  determinant  of  performance,  knowledge  and  skill  are  not. 
Military  motives  is  again  a  direct  determinant  of  performance,  which  was  not  unexpected  due  to 
the  nature  of  the  criterion.  Attitudes  such  as  job  satisfaction  and  affective  commitment  should 
lead  directly  to  behaviors  relevant  to  organizational  commitment,  without  having  to  go  through 
goal  setting  or  expectancy  cognitions.  Self-regulation  makes  sense  as  a  direct  determinant 
because  behaviors  such  as  following  orders  or  obeying  rules  may  be  difficult  to  perform 
sometimes  and  self-regulation  could  be  required  to  keep  the  behavior  on  track.  Knowledge  and 
skill  are  probably  not  very  meaningful  constructs  with  respect  to  organizational  commitment. 
These  types  of  behaviors  do  not  really  require  any  specific  skill  to  perform  them.  Knowledge  is 
relevant  to  the  extent  that  individuals  need  to  know  the  policies,  rules,  and  values  of  the 
organization  to  behave  in  accordance  with  them,  but  these  are  probably  so  central  to  military  life 
that  there  would  be  little  opportunity  for  differential  knowledge  levels  to  influence  behavior. 

Experience  is  considered  an  indirect  performance  determinant,  but  in  this  model  it  had  a 
direct  path  to  performance.  A  possible  explanation  for  this  is  that  commitment  to  ROTC  is 
naturally  related  to  experience  because  cadets  and  midshipmen  who  are  not  committed  are 
weeded  out  as  they  leave  the  program.  Therefore,  those  who  have  more  experience  will  tend  to 
be  rated  higher  on  organizational  commitment  just  by  virtue  of  having  stayed  with  the  program, 
and  this  relationship  would  not  go  through  a  mediating  variable.  If  this  explanation  is  correct,  the 
direct  path  from  experience  to  organizational  commitment  performance  probably  would  not  be 
found  in  a  sample  of  junior  commissioned  officers  in  their  first  active  duty  service  obligation. 
These  officers  have  a  3-,  4-,  or  5-year  commitment,  depending  on  commissioning  source,  that 
must  be  met  before  they  are  eligible  to  leave,  so  experience  would  not  be  a  good  direct  indicator 
of  commitment. 
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It  is  interesting  that  the  models  for  organizational  commitment  and  maintaining  good 
working  relationships  are  consistent  with  the  counterproductive  work  behavior  (CWB)  model  for 
which  Mount,  Ilies,  and  Johnson  (2006)  found  support.  CWB  is  any  intentional  behavior  on  the 
part  of  an  employee  that  is  contrary  to  the  legitimate  interests  of  the  organization  (Sackett  & 
DeVore,  2001 ).  Examples  of  CWBs  include  theft;  property  destruction;  misuse  of  information, 
time,  or  resources;  drug  or  alcohol  use;  unsafe  behavior;  poor  attendance;  intentionally  slow  or 
sloppy  work;  and  inappropriate  verbal  or  physical  actions  (Gruys,  2000).  Mount  et  al. 
distinguished  between  interpersonal  CWBs,  which  are  behaviors  directed  at  individuals  in  the 
organization  with  the  intent  to  produce  emotional  or  physical  discomfort  or  harm,  and 
organizational  CWBs,  which  are  behaviors  directed  toward  harming  the  interests  of  the 
organization.  Although  not  the  same  as  the  performance  dimensions  we  studied,  maintaining 
good  working  relationships  and  organizational  commitment  should  be  highly  negatively 
correlated  with  interpersonal  CWB  and  organizational  CWB,  respectively.  Mount  et  al.  found 
support  for  a  model  in  which  job  satisfaction  partially  mediated  the  relationships  between 
personality  traits  and  CWB.  Job  satisfaction  was  related  to  both  types  of  CWB.  Agreeableness 
had  a  direct  effect  on  interpersonal  CWB  as  well  as  an  indirect  effect  on  both  types  of  CWB 
through  job  satisfaction.  Conscientiousness  had  a  direct  path  to  organizational  CWB,  but  the 
mediating  effect  through  job  satisfaction  was  weaker.  In  our  research,  we  found  that 
agreeableness  had  an  indirect  effect  on  maintaining  good  working  relationships  through  both 
military  motives  (which  includes  job  satisfaction)  and  knowledge.  In  the  organizational 
commitment  model,  the  agreeableness  facet  of  trust  had  an  indirect  influence  through  military 
motives.  Conscientiousness  also  had  an  indirect  influence  through  military  motives  and  self¬ 
regulation. 

Showing  Initiative 

The  best- fitting  model  for  showing  initiative  was  similar  to  the  best-fitting  models  for  the 
other  two  citizenship  performance  dimensions.  In  all  three  models,  military  motives  was  a  direct 
determinant  of  performance,  while  expectancies  and  work  habits  were  not.  Besides  military 
motives,  each  citizenship  performance  dimension  had  one  other  direct  determinant — knowledge 
for  maintaining  good  working  relationships,  self-regulation  for  organizational  commitment,  and 
skill  for  showing  initiative.  Even  the  indirect  determinants  had  similar  relationships.  Cognitive 
ability  was  a  positive  determinant  of  knowledge  and  a  negative  determinant  of  interests. 
Experience  was  a  determinant  of  expectancies  and  interests  in  all  three  models,  and  of  skill  in  the 
maintaining  good  working  relationships  and  showing  initiative  models.  Conscientiousness  was 
included  in  the  models  for  organizational  commitment  and  showing  initiative,  and  in  both 
models  it  was  a  determinant  of  knowledge,  expectancies,  self-regulation,  and  military  motives. 

The  similarity  between  these  three  models  suggests  that  a  single  model  for  citizenship 
performance  may  be  appropriate.  Citizenship  performance  involves  behaviors  that  are  typically 
not  formally  required  on  the  job  (although  they  may  be;  Organ,  1997)  and  that  support  the 
organization  without  necessarily  providing  a  direct  benefit  to  the  individual.  Given  this 
definition,  it  is  not  surprising  that  the  general  motives  of  job  satisfaction,  affective  commitment, 
and  military  values  are  direct  determinants  of  citizenship  performance.  An  individual  who  is 
dissatisfied  with  his  or  her  job,  does  not  have  an  affective  attachment  to  the  organization,  and 
does  not  share  the  values  of  the  organization  would  be  very  unlikely  to  engage  in  behaviors  such 
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as  participating  in  social  activities  arranged  for  the  unit,  exceeding  standards  when  carrying  out 
orders,  or  performing  extra  duties  without  being  asked. 

This  lends  support  to  Organ  and  Ryan’s  (1995)  idea  that  the  relationship  between 
personality  and  organizational  citizenship  behaviors  (OCB)  is  probably  mediated  by  attitudes 
such  as  job  satisfaction,  organizational  commitment,  and  fairness  perceptions.  They  came  to  this 
conclusion  based  on  the  observation  that  job  attitudes  tend  to  be  more  strongly  related  to  OCB 
than  are  personality  variables  (Podsakoff  et  al.,  2000).  This  research  empirically  demonstrates 
this  mediating  effect  of  attitudes  on  the  relationship  between  personality  and  citizenship 
performance. 

Other  determinants  of  citizenship  performance  appear  to  depend  on  the  behaviors 
representing  the  dimension  being  predicted.  Some  behaviors,  such  as  those  included  in  the 
organizational  commitment  dimension,  do  not  require  any  particular  knowledge  or  skill  in  order 
to  enact  them  (e.g.,  following  orders,  displaying  a  professional  appearance  while  in  uniform),  so 
knowledge  and  skill  are  unlikely  to  be  direct  determinants.  Knowledge  was  a  determinant  of 
maintaining  good  working  relationships,  because  knowledge  of  how  to  act  in  social  situations 
varies  considerably  among  individuals  (Schneider  &  Johnson,  2005).  Social  skill  would  be  a 
likely  determinant  of  certain  types  of  social  behavior,  such  as  counseling,  because  knowing  what 
to  do  can  be  very  different  from  actually  being  able  to  do  it  well.  Skill  was  a  determinant  of 
showing  initiative,  because  behaviors  like  seeking  ways  to  accomplish  work  when  there  is  no 
clear  solution,  anticipating  the  needs  of  the  unit,  and  initiating  projects  that  may  contribute  to 
mission  success  require  a  certain  amount  of  skill.  If  showing  initiative  were  only  measured  using 
behaviors  such  as  volunteering,  seeking  clarification,  and  finding  additional  work  when  one’s 
own  duties  are  completed,  motivation  alone  would  likely  determine  performance  because  no 
particular  knowledge  or  skill  is  necessary  to  engage  in  those  behaviors. 

Adapting  to  Uncertain  or  Changing  Situations 

The  best-fitting  model  for  adaptive  performance  was  similar  to  the  model  for  decision 
making/problem  solving,  but  there  were  also  some  important  differences.  As  with  decision 
making/problem  solving,  expectancies  was  a  direct  determinant  of  performance,  with  goal 
commitment  acting  as  a  suppressor  variable  that  strengthened  this  relationship.  Knowledge  was 
also  a  direct  determinant.  Unlike  decision  making/problem  solving,  skill  and  self-regulation  were 
not  significant  determinants  of  performance. 

This  was  the  only  performance  dimension  for  which  there  was  a  direct  path  from  a 
personality  construct  to  performance.  Trust  had  a  direct  path  to  performance  as  well  as  a  path 
through  military  motives.  We  examined  the  items  measuring  trust  and  the  items  measuring 
adaptive  performance  to  determine  why  trust  would  be  directly  related  to  performance  rather 
than  going  through  some  other  construct  that  is  theoretically  more  proximal  to  performance. 
Adaptive  performance  in  the  military  often  involves  giving  direction  to  others,  and  several  of  the 
items  in  the  performance  rating  instrument  measure  that  aspect  of  adapting  (e.g.,  provides  quick, 
clear,  and  effective  direction  when  faced  with  changing  situations  or  unexpected  events;  provides 
clear  structure  to  others  in  uncertain  or  ambiguous  situations).  Other  items  do  not  explicitly 
mention  directing  others  (e.g.,  responds  quickly  to  difficult  situations)  but  are  likely  to  involve 
directing  others  in  many  situations.  Trust  measures  the  extent  to  which  the  respondent  trusts  and 
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believes  in  others.  In  order  to  effectively  adapt  to  uncertain  or  changing  situations  that  require 
direction  to  be  given  to  others,  it  is  necessary  to  trust  that  others  will  immediately  and  effectively 
carry  out  the  orders  given  to  them.  If  the  individual  does  not  trust  others  to  do  what  needs  to  be 
done,  he  or  she  may  be  overwhelmed  and  ineffective  by  trying  to  do  everything  on  his  or  her 
own.  Thus,  knowledge,  skill,  and  motivation  may  not  be  enough  if  the  personality  characteristic 
of  trusting  others  does  not  allow  for  quick  orders  to  be  given  to  others  when  changes  need  to  be 
made.  We  suspect  that  skill  would  be  a  likely  mediating  variable  for  trust  if  this  aspect  of 
adaptive  skill  had  been  measured  by  our  skill  measure. 

Summary 

The  Johnson  (2003)  model  was  partially  supported  for  each  performance  dimension 
investigated,  but  there  were  important  differences  in  each  case.  The  primary  difference  between 
the  Johnson  model  and  the  best-fitting  models  was  that  work  habits  had  no  influence  on 
performance  or  self-regulation.  Another  difference  was  that  each  component  of  motivation 
(motives,  proactive  cognitions,  and  self-regulation)  had  the  potential  to  be  a  direct  determinant  of 
performance,  rather  than  there  being  a  completely  mediated  relationship  between  the  components 
and  performance.  Other  than  those  two  aspects,  the  model  was  supported  in  that  personality, 
ability,  and  experience  influenced  performance  through  two  or  more  of  the  hypothesized 
mediating  variables  (except  for  the  direct  effect  of  trust  on  adaptive  performance).  There  was 
also  support  for  the  notion  that  personality  variables  have  differential  relationships  with  different 
components  of  motivation,  as  suggested  by  Heckhausen  and  Kuhl  (1985).  This  supports  the  need 
for  splitting  motivation  into  its  components  in  order  to  truly  understand  how  personality 
influences  performance. 

As  expected,  the  precise  nature  of  the  relationships  between  predictors,  mediators,  and 
criteria  depended  on  the  performance  dimension  and  predictors  included  in  the  model.  The 
model  that  was  most  similar  to  the  general  Johnson  (2003)  model  was  the  one  for  the  decision 
making/problem  solving  performance  dimension,  which  was  the  only  dimension  of  task 
performance  included  in  this  project.  For  this  dimension,  knowledge,  skill,  proactive  cognitions, 
and  self-regulation  were  all  direct  determinants  of  performance,  and  each  of  these  determinants 
was  predicted  by  different  combinations  of  personality  traits,  experience,  and  motives.  Motives 
were  related  to  performance  only  through  the  mediating  influence  of  proactive  cognitions.  A 
similar  model  was  supported  for  the  adaptive  performance  dimension  of  adapting  to  uncertain  or 
changing  situations.  The  differences  were  that  only  some  of  the  hypothesized  direct  determinants 
had  significant  paths  to  performance,  and  one  personality  trait  had  a  direct  influence  on 
performance. 

This  model  did  not  generalize  to  the  citizenship  performance  dimensions,  although  a 
similar  model  was  found  for  each  different  dimension  of  citizenship  performance.  In  this  model, 
motives  were  a  direct  determinant  of  performance  and  proactive  cognitions  were  not. 

Knowledge,  skill,  and  self-regulation  were  direct  determinants,  but  never  all  at  once  and  the  one 
that  was  a  determinant  depended  on  the  citizenship  dimension  of  interest.  There  were  also  two 
cases  in  which  a  hypothesized  indirect  performance  determinant  had  a  direct  influence  on 
performance. 
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To  summarize,  modified  versions  of  the  Johnson  (2003)  model  were  found  to  describe 
the  process  by  which  predictor  variables  influence  performance.  Different  models  are  necessary 
to  describe  the  process  for  task,  citizenship,  and  adaptive  performance. 

Contributions  of  This  Research 

This  research  advances  our  understanding  of  job  performance  prediction  by  providing 
evidence  in  support  of  a  model  of  the  process  by  which  individual-difference  variables  predict 
specific  dimensions  of  performance.  Many  models  of  the  relationship  between  individual 
differences  and  job  performance  have  been  proposed,  but  there  has  been  very  little  empirical 
support  for  any  of  them.  This  is  especially  true  for  less-studied  aspects  of  performance  such  as 
citizenship  performance  and  adaptive  performance,  which  were  the  primary  focus  of  this 
research.  An  established  model  of  the  process  by  which  individual  differences  in  predictor 
variables  lead  to  individual  differences  in  specific  dimensions  of  performance  can  be  used  to 
guide  research  linking  specific  predictors  to  specific  performance  dimensions  by  helping  to 
identify  theoretically  relevant  predictors  for  different  criteria. 

This  research  advances  our  understanding  of  the  performance  prediction  process.  No 
previous  research  on  performance  prediction  models  has  included  this  many  predictors,  this 
many  criteria,  or  this  many  mediators.  Tests  of  models  of  performance  have  typically  included 
measures  of  job  knowledge  and  skill  (or  job  proficiency),  but  these  studies  only  measure 
motivation  indirectly.  For  example,  Borman  et  al.  (1995)  included  measures  of  personality  as  a 
surrogate  for  motivation.  Lance  and  Bennett  (2000)  included  measures  of  awards  and 
disciplinary  actions  as  surrogates  for  motivation.  Schneider  and  Johnson  (2005)  measured 
motivation  directly,  but  only  in  terms  of  the  importance  of  engaging  in  certain  behaviors  to  the 
individual.  Barrick  et  al.  (2002)  developed  a  measure  of  three  psychological  motives 
(accomplishment  striving,  status  striving,  communion  striving)  and  demonstrated  that  these 
motives  mediated  the  relationship  between  personality  and  sales  performance.  They  did  not 
include  measures  of  knowledge  or  skill  in  their  study.  In  our  research,  we  measured  motivation 
in  several  different  ways.  We  developed  a  measure  of  self-regulation  success  and  demonstrated 
that  it  was  related  to  motivational  traits,  action  control,  and  performance.  We  developed  a 
proactive  cognitions  scale  that  included  measures  of  expectancies,  self-efficacy,  and  goal 
commitment.  We  measured  the  following  potential  motives  for  engaging  in  performance¬ 
relevant  behavior:  (a)  job  satisfaction,  (b)  military  values,  (c)  affective  commitment,  (d)  interests, 
(e)  prosocial  values,  (f)  organizational  concern,  and  (g)  impression  management. 

The  tests  of  performance  models  mentioned  above  have  been  limited  to  task  performance 
(e.g.,  technical  proficiency,  sales  performance)  or  overall  performance  as  a  criterion.  Our 
research  tested  the  performance  model  on  five  different  dimensions  of  performance  representing 
the  task,  citizenship,  and  adaptive  performance  domains. 

There  have  been  only  a  few  studies  that  have  attempted  to  test  the  idea  that  citizenship 
knowledge  and  skill  mediate  the  relationship  between  personality  and  citizenship  performance. 
Schmit,  Motowidlo,  Degroot,  Cross,  and  Kiker  (1996)  investigated  the  mediating  role  of 
citizenship  knowledge  in  the  personality-citizenship  performance  relationship  in  a  sample  of 
sales  associates.  In  this  research,  the  personality  measures  assessed  extraversion,  agreeableness, 
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and  conscientiousness.  The  citizenship  knowledge  measure  was  a  situational  interview  designed 
to  assess  the  participants’  knowledge  of  appropriate  customer  service  behaviors;  the  job 
performance  measure  was  supervisor  ratings  of  the  participants’  customer  service-related  job 
performance.  Schmit  et  al.  found  that  citizenship  knowledge  mediated  the  personality- 
citizenship  job  performance  relationship  only  in  the  case  of  extraversion.  It  is  not  clear  how 
relevant  this  research  is,  however,  because  customer  service  is  probably  a  combination  of  task 
and  citizenship  performance. 

Schneider  and  Johnson  (2001)  used  a  situational  judgment  test  as  a  measure  of 
citizenship  knowledge  for  the  dimensions  of  personal  support  and  conscientious  initiative. 
Predictors  were  agreeableness,  achievement,  dependability,  and  cognitive  ability.  Criteria  were 
supervisor  ratings  of  personal  support,  conscientious  initiative,  and  customer  service 
performance.  The  citizenship  knowledge  mediation  hypothesis  was  tested  separately  for  each 
construct,  and  support  was  mixed.  Conscientious  initiative  knowledge  did  not  mediate  the 
relationship  between  achievement  and  conscientious  initiative  performance.  When  testing  the 
relationship  between  agreeableness  and  personal  support  performance,  a  model  in  which 
personal  support  knowledge  mediated  the  relationship  fit  equally  as  well  as  a  model  in  which 
there  was  no  mediation.  For  customer  service,  the  mediation  effect  was  found  for  achievement, 
but  not  for  dependability  or  agreeableness. 

Schneider  and  Johnson  (2005)  demonstrated  that  a  constructed  response  video-based  test 
of  social  knowledge  mediated  the  relationship  between  a  “social  cunning”  personality  factor  and 
interpersonal  sensitivity  performance,  but  it  did  not  mediate  the  relationship  between  personality 
and  performance  for  four  other  performance  dimensions  (although  it  did  mediate  the  relationship 
between  social  insight  and  performance  for  three  dimensions). 

In  this  research,  we  tested  the  mediating  role  of  citizenship  knowledge  and  skill  between 
personality  and  performance  for  three  citizenship  performance  dimensions,  in  the  presence  of 
other  potential  mediators.  We  found  that  knowledge  mediated  the  relationship  between 
agreeableness  and  maintaining  good  working  relationships  performance  and  that  skill  mediated 
the  relationship  between  extraversion  and  showing  initiative  performance.  Neither  skill  nor 
knowledge  was  a  mediator  for  organizational  commitment  performance.  By  including  other 
potential  mediators  in  our  model,  we  demonstrated  that  citizenship  knowledge  and  skill  mediate 
some  personality-performance  relationships,  but  other  personality  variables  predict  citizenship 
performance  through  motivation. 


Limitations 

This  research  does  have  several  potential  limitations.  First,  several  of  the  measures  of  our 
constructs  were  developed  specifically  for  this  project  and  are  experimental  in  nature.  Although 
we  presented  evidence  to  support  the  construct  validity  of  these  measures,  it  is  still  not  clear  how 
well  these  measures  operationalized  the  constructs.  The  self-regulation  and  work  habits  measures 
were  especially  experimental  because  no  one  had  attempted  to  measure  these  constructs  in  this 
way  before.  The  Self-Regulation  Scale  had  good  psychometric  properties,  but  the  Work  Habits 
Scale  was  uncorrelated  with  performance  and  the  extent  to  which  this  is  due  to  the  nature  of  the 
criterion  versus  inadequate  measurement  is  not  clear. 
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The  Past  Behavior  Record  was  used  as  a  measure  of  skill  because  it  could  be 
administered  to  a  large  group  of  people  along  with  many  other  instruments  in  a  timely  fashion. 
Although  it  was  correlated  with  decision  making/problem  solving  and  showing  initiative,  it  was 
not  correlated  with  maintaining  good  working  relationships,  organizational  commitment,  or 
adaptive  performance.  This  may  be  due  to  the  nature  of  the  criterion  (e.g.,  skill  at  displaying 
organizational  commitment  probably  does  not  make  sense),  but  skill  could  be  measured  in  a 
better  way  given  more  time  and  resources.  For  example,  a  structured  interview  would  be 
preferred  over  a  PBR  because  the  interviewer  could  probe  to  make  sure  the  individual  provides 
all  the  necessary  information  to  evaluate  the  level  of  skill  he  or  she  possesses.  An  even  better 
measure  would  be  a  role-play  exercise  to  evaluate  interpersonal  skill,  or  a  computerized 
simulation  to  evaluate  adaptability. 

A  second  limitation  is  the  sample,  which  was  composed  of  ROTC  cadets  and 
midshipmen  across  different  military  branches  at  different  universities.  It  would  be  preferable  to 
stay  within  one  branch  to  eliminate  any  possible  confounding  effects  of  branch,  but  that  would 
have  severely  limited  the  sample  size  we  were  able  to  obtain.  Also,  although  ROTC  cadets  are  in 
training  to  become  junior  commissioned  officers,  there  is  some  question  as  to  the  extent  to  which 
these  results  will  generalize  to  actual  junior  commissioned  officers. 

Finally,  our  sample  size  was  fairly  low  relative  to  the  number  of  parameters  in  our 
structural  equation  models.  Although  our  models  fit  well,  a  larger  sample  size  would  lead  to 
more  confidence  in  the  results.  Some  nonsignificant  path  coefficients  also  may  also  have  reached 
standard  significance  levels  with  a  larger  sample. 

Directions  for  Future  Research 

There  are  a  number  of  areas  of  future  research  suggested  by  this  project,  some  of  which 
can  be  conducted  using  the  data  already  collected.  One  area  of  research  we  are  planning  is  the 
extent  to  which  different  methods  of  scoring  SJTs  produce  comparable  scores,  reliabilities,  and 
criterion-related  validities.  We  did  some  of  this  research  already  to  determine  that  we  would  use 
the  cadre  scoring  key,  but  there  is  great  potential  for  additional  scoring  systems  to  explore.  The 
different  methods  we  will  examine  are  (a)  scoring  based  on  a  correct-incorrect  coding  of  the 
“best”  response  only,  (b)  scoring  based  on  the  mean  effectiveness  rating  of  the  response  chosen, 
(c)  scoring  based  on  selection  of  both  a  “best”  and  a  “worst”  response,  and  (d)  scoring  based  on 
rating  the  effectiveness  level  of  each  response.  We  will  evaluate  each  of  these  scoring  procedures 
for  (a)  consensus-based  scoring,  (b)  expert  based  scoring,  and  (c)  empirical  scoring.  Consensus- 
based  scoring  also  allows  us  to  examine  differences  in  effectiveness  ratings  associated  with 
subgroups  of  examinees,  so  we  will  determine  if  there  are  meaningful  differences  in  the 
appropriate  situational  responses  across  different  ROTC  branches  or  across  third-  and  fourth- 
year  cadets/midshipmen. 

Another  area  of  research  we  will  explore  is  the  comparison  of  the  conventional  rating 
format  to  the  MARS  rating  format  for  the  PBR.  We  will  investigate  a  variety  of  mechanical 
scoring  schemes  for  the  MARS  ratings  to  determine  the  optimal  combination  of  situation, 
behavior,  and  outcome  for  each  performance  dimension.  We  will  compare  the  validity  of  the 
mechanical  scoring  system  for  MARS  with  the  clinical  scoring  system  used  by  raters  in  arriving 
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at  overall  judgments  using  the  conventional  format.  We  expect  that  the  MARS  format  will  result 
in  better  criterion  prediction.  This  would  suggest  an  easier  and  more  valid  way  of  evaluating 
interviewees  in  structured  interviews. 

One  implication  of  the  performance  models  we  identified  is  that  narrow  personality  traits 
should  be  better  predictors  of  specific  performance  dimensions  than  should  broad  personality 
variables  such  as  the  Big  Five.  The  determinants  of  performance  are  necessarily  at  a  more 
specific  level  than  is  the  performance  dimension  itself,  and  prediction  is  better  when  constructs 
are  matched  on  level  of  specificity  (Schneider,  Hough,  &  Dunnette,  1996).  The  data  collected  in 
this  research  and  the  support  for  the  models  tested  can  be  used  to  explain  the  predictive 
superiority  of  narrow  personality  traits  by  demonstrating  that  they  have  stronger  relationships 
with  the  determinants  of  performance.  For  example,  the  anger  facet  of  emotional  stability  was 
expected  to  influence  more  determinants  of  maintaining  good  working  relationships  than  was  the 
anxiety  facet.  This  can  be  demonstrated  by  examining  the  correlations  of  each  facet  with  the 
different  variables  that  have  been  found  to  influence  performance.  Facets  that  are  correlated  with 
more  determinants  of  performance  should  have  larger  correlations  with  performance;  and  when 
this  happens,  the  facet  score  also  should  be  a  better  predictor  than  the  factor  score. 

Another  direction  for  future  research  is  to  study  how  potential  moderator  variables 
influence  the  relationships  between  performance  and  its  determinants.  Some  of  the  moderators 
that  have  been  found  to  influence  the  extent  to  which  personality  predicts  performance  are 
situational  strength  (Beaty,  Cleveland,  &  Murphy,  2001),  occupation  (Barrick,  Mount,  &  Judge, 
2001),  time  on  job  (Helmreich,  Sawin,  &  Carsrud,  1986),  autonomy  (Gellatly  &  Irving,  2001), 
and  typical  vs.  maximum  performance  measurement  (Marcus,  Goffin,  Johnston,  &  Rothstein, 
2007).  Personality  and  ability  have  been  found  to  interact  when  predicting  performance  (Wright, 
Kacmar,  McMahan,  &  Deleeuw,  1995),  although  most  recent  studies  have  shown  no  interaction 
(Mount,  Barrick,  &  Strauss,  1999;  Sackett,  Gruys,  &  Ellingson,  1998). 

We  remain  convinced  that  work  habits  can  directly  influence  performance  in  some 
situations  even  though  that  was  not  found  for  the  performance  dimensions  we  studied.  Work 
habits  should  influence  performance  directly  when  job-relevant  behavior  occurs  automatically 
despite  motivation  to  behave  otherwise.  Autonomy  is  expected  to  moderate  this  direct 
relationship  because  work  habits  will  have  less  of  an  influence  on  performance  in  stronger 
situations.  Autonomy  is  the  extent  to  which  the  environment  allows  an  individual  to  behave  in 
idiosyncratic  ways.  The  stronger  a  situation,  the  less  autonomy  the  individual  has.  The  ROTC 
environment  may  be  too  strong  a  situation  for  work  habits  to  be  related  to  performance,  so  future 
research  could  examine  this  relationship  in  an  environment  that  clearly  allows  for  the  operation 
of  work  habits.  In  addition,  the  criterion  must  consist  of  behaviors  that  can  occur  automatically. 
Given  a  criterion  like  this — the  opportunity  for  work  habits  to  influence  performance  and 
variance  in  the  extent  to  which  individuals’  work  habits  are  consistent  with  performance — we 
expect  that  work  habits  influence  performance  and  self-regulation  in  the  manner  hypothesized. 

As  mentioned  in  the  section  on  limitations,  future  research  should  explore  alternative 
ways  of  measuring  skill,  especially  for  constructs  such  as  maintaining  good  working 
relationships  and  adaptive  performance.  Structured  interviews,  work  sample  tests,  role  plays,  and 
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computerized  simulations  are  alternative  ways  of  measuring  skills  that  probably  would  provide 
better  measurement  than  the  PBR. 

This  type  of  research  should  continue  to  be  conducted  with  different  performance 
dimensions.  The  performance  models  for  the  citizenship  performance  dimensions  differed 
considerably  from  the  task  and  adaptive  performance  models,  but  they  were  relatively  consistent 
across  the  three  citizenship  performance  dimensions  included  in  this  project.  It  would  be 
beneficial  to  test  this  model  on  other  citizenship  performance  dimensions  to  determine  if  there  is 
a  general  model  of  citizenship  performance  that  can  be  distinguished  from  a  model  for  task 
performance  or  for  adaptive  performance.  This  research  also  can  be  conducted  on  many  other 
task  performance  dimensions  besides  the  one  we  included  in  this  project.  In  addition,  research  on 
additional  dimensions  of  adaptive  performance  (Pulakos  et  al.,  2000)  would  help  to  distinguish 
adaptive  performance  dimensions  from  task  and  citizenship  performance. 

Another  type  of  performance  that  was  not  addressed  in  this  research  is  counterproductive 
work  behavior  (CWB).  Models  of  the  relationships  between  individual  differences  and  CWBs 
(e.g.,  Cullen  &  Sackett,  2003;  Mount  et  al.,  2006)  are  generally  consistent  with  the  citizenship 
performance  model  for  which  we  found  support,  so  extending  this  research  to  CWBs  is  a  natural 
next  step. 

Finally,  this  research  should  be  replicated  in  different  samples.  We  used  an  ROTC 
sample  because  they  are  in  training  to  become  junior  commissioned  officers  and  we  expected  the 
results  to  generalize  to  that  population.  Future  research  should  be  conducted  in  a  sample  of  actual 
junior  commissioned  officers  to  ensure  that  these  results  are  consistent  across  samples.  Enlisted 
Soldiers,  NCOs,  and  senior  officers  are  other  potential  military  samples;  the  model  also  can  be 
tested  in  civilian  samples  for  different  types  of  jobs. 

Potential  Army/Military  Applications 

This  research  provides  the  Army  with  a  tool  that  can  be  used  to  identify  the  constructs 
through  which  individual  difference  variables  work  to  influence  performance  on  specific 
dimensions.  This  is  especially  important  for  citizenship  and  adaptive  performance,  which  are 
important  components  of  Army  officer  jobs  but  have  received  little  research  attention  in  a 
military  or  civilian  context.  This  model  contributes  to  a  better  understanding  of  the  relationships 
between  predictors,  mediators,  and  job  performance  criteria.  The  Army  can  use  this  model  to 
identify  interventions  that  will  have  the  greatest  impact  on  areas  of  performance  that  are  deficient 
in  certain  officers.  For  example,  attempts  to  increase  citizenship  performance  behaviors  should 
be  directed  at  increasing  job  satisfaction.  This  tool  also  would  be  effective  in  identifying  training 
and/or  development  needs.  Given  a  criterion  construct  on  which  an  individual’s  performance  is 
in  need  of  improvement,  this  model  can  help  to  identify  the  determinants  of  performance  on  that 
construct.  For  example,  an  individual  possessing  adequate  skill  and  knowledge  may  determine 
that  he  or  she  must  learn  new  self-regulatory  strategies  to  maintain  goal-directed  behavior. 

Several  products  that  may  be  very  useful  to  the  Army  were  developed  for  this  project. 
First,  the  various  motivation-related  instruments  we  developed/assembled  and  validated  against 
criteria  of  importance  to  the  Army  should  prove  to  be  quite  useful  as  self-development  tools. 
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Completing  these  instruments  can  help  Soldiers  and/or  officers  learn  about  their  own  attitudes, 
values,  goal  commitment,  self-efficacy,  self-regulatory  skills,  and  work  habits  relevant  to  five 
critical  performance  dimensions. 

Second,  the  Past  Behavior  Record  Form  may  prove  to  be  useful  for  training.  In  particular, 
the  evaluation  guide  used  to  score  this  instrument  will  provide  useful  information  regarding 
where  various  skill-relevant  behaviors  fall  on  various  performance  continua.  Moreover,  the 
principles  induced  during  the  process  of  calibrating  the  performance-relevant  skills  to  different 
points  on  these  anchored  rating  scales  will  make  useful  learning  points  in  training  programs.  The 
situational  judgment  test  also  will  be  a  useful  self-paced  training  tool.  Individuals  are  presented 
with  realistic  situations  and  choose  from  a  variety  of  alternative  response  options.  Explanations 
of  the  strengths  and  weaknesses  of  each  response  can  be  developed  to  help  the  trainee  understand 
the  level  of  effectiveness  of  the  chosen  option  and  the  most  effective  option. 

Finally,  the  performance  rating  instrument  may  have  applied  value  for  performance 
appraisal,  training  needs  analysis,  or  criteria  for  future  validation  studies.  In  addition,  it  would 
make  a  useful  feedback  and  development  tool.  The  performance  rating  form  is  a  multi-source 
rating  tool.  As  such,  it  cannot  only  provide  a  profile  of  strengths  and  weaknesses,  it  can  provide 
information  about  discrepancies  between  how  an  individual  is  seen  from  different  perspectives 
(i.e.,  subordinates,  peers,  superiors).  It  would  be  extremely  useful,  for  example,  for  a  junior 
officer  to  become  aware  of  differences  between  his  or  her  self-perceived  performance  and  the 
perceptions  of  his  or  her  performance  held  by  subordinates. 
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Performance  Dimension  Definitions 


Broad  Performance  Categories 


Task  Performance 

Task  performance  includes  the  core  activities  that  distinguish  between  one  job  and 
another  and  are  typically  the  kinds  of  activities  emphasized  by  formal  job  descriptions.  Task 
performance  consists  of  activities  that  contribute  to  the  organization’s  technical  core  either  (a) 
directly  by  creating  the  goods  or  delivering  the  services  produced  by  the  organization,  or  (b) 
indirectly  by  providing  it  with  needed  materials  or  services  (e.g.,  accounting,  distribution, 
personnel,  legal).  This  includes  management  functions  such  as  planning,  coordinating, 
organizing,  and  supervising  to  ensure  that  the  activities  of  others  are  carried  out  effectively  and 
efficiently. 

Citizenship  Performance 

Citizenship  performance  consists  of  activities  that  support  the  broader  environment  in 
which  the  technical  core  must  function,  including  behaviors  such  as  volunteering  for  tasks  not 
formally  part  of  the  job,  demonstrating  effort,  helping  and  cooperating  with  others,  following 
organizational  rules  and  procedures,  and  supporting  organizational  objectives.  Citizenship 
performance  is  less  likely  than  task  performance  to  be  role-prescribed,  although  in  some  jobs 
certain  citizenship  performance  dimensions  may  be  required.  Citizenship  performance  activities 
are  typically  common  to  many  or  all  jobs. 

Adaptive  Performance 

Adaptive  performance  is  the  proficiency  with  which  a  person  alters  his  or  her  behavior  to 
meet  the  demands  of  the  environment,  an  event,  or  a  new  situation.  Many  aspects  of  adaptive 
performance  overlap  with  components  of  task  or  citizenship  performance,  but  one  aspect  that  is 
distinct  is  dealing  with  uncertain  and  unpredictable  work  situations.  This  includes  taking  action 
when  necessary  without  having  all  the  facts  at  hand;  adjusting  plans,  actions,  or  priorities  to  deal 
with  changing  situations;  and  imposing  structure  to  provide  focus  in  dynamic  situations. 

Specific  Task  Performance  Dimensions 
Job-Specific  Task  Proficiency 

This  is  the  degree  to  which  an  individual  can  perform  the  tasks  that  are  central  to  the  job. 
These  tasks  represent  the  core  of  the  job  and  are  the  primary  definers  of  the  job.  Examples 
include  driving  a  tank,  directing  air  traffic,  and  building  bridges. 
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Non-Job-Specific  Task  Proficiency 

This  is  the  degree  to  which  an  individual  can  perform  tasks  or  behaviors  that  are  not 
specific  to  his  or  her  particular  job.  In  the  military,  for  example,  there  are  sets  of  common  tasks 
(e.g.,  first  aid,  basic  navigation,  using  protective  equipment)  for  which  everyone  is  responsible. 

Written  and  Oral  Communication  Proficiency 

This  reflects  the  proficiency  with  which  someone  can  write  or  speak  when  making  oral  or 
written  presentations  for  audiences  varying  from  one  person  to  thousands.  This  would  include 
presenting  an  idea  in  a  meeting,  writing  a  proposal,  or  doing  a  presentation  for  clients. 

M  anagement/ Ad  min  istration 

This  includes  the  major  elements  of  management  that  are  distinct  from  direct  supervision, 
such  as  performance  of  behaviors  directed  at  articulating  goals  for  the  unit  or  enterprise, 
organizing  people  and  resources  to  work  on  them,  monitoring  progress,  helping  to  solve 
problems  or  overcome  crises  that  stand  in  the  way  of  goal  accomplishment,  controlling 
expenditures,  obtaining  additional  resources,  and  representing  the  unit  in  dealings  with  other 
units. 


Decision  Making  Making  sound  and  timely  decisions;  paying  attention  to  and  taking  into 
account  all  relevant  information  in  making  decisions. 


Problem  Solving  Generating  creative  alternatives  to  solve  problems;  integrating  seemingly 
unrelated  information  and  developing  effective  solutions;  anticipating 
possible  problems. 


Planning  and  Formulating  short-  and  long-term  goals  and  objectives;  forecasting  possible 
Organizing  problems  for  the  unit/organization  and  developing  strategies  for  addressing 

these  problems;  organizing  and  prioritizing  work;  time  management. 


Coordinating 

Resources 

Administration 
and  Paperwork 


Properly  utilizing  personnel  and  other  resources  to  increase  unit  and 
organizational  effectiveness;  coordinating  the  work  within  the  unit;  balancing 
interests  of  own  unit  and  those  of  the  organization  as  a  whole,  if  necessary. 

Handling  paperwork  requirements;  performing  day-to-day  administrative 
tasks  such  as  reviewing  reports,  going  through  mail,  approving  routine 
requests,  and  so  on;  keeping  accurate  records;  and  administering  policies  as 
appropriate. 


Supervision 

This  includes  all  the  behaviors  directed  at  influencing  the  performance  of  supervisees 
through  face-to-face  interpersonal  interaction  and  influence.  Supervisors  set  goals  for  their 
supervisees,  teach  them  more  effective  methods,  model  the  appropriate  behaviors,  and  reward  or 
punish  in  appropriate  ways. 
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Guiding,  directing, 
and  motivating 
subordinates  and 
providing  feedback 


Providing  guidance  and  direction  to  subordinates;  motivating  subordinates 
by  providing  them  with  recognition,  encouragement,  constructive  criticism, 
and  other  feedback  as  appropriate;  helping  to  set  goals  and  maintaining 
performance  standards  for  subordinates;  monitoring  subordinate 
performance. 


Specific  Citizenship  Performance  Dimensions 


Personal  Support 


Maintaining 
Good  Working 
Relationships 

Interpersonal 

Adaptability 


Helping  Others 


Cooperation 


Showing 

Consideration 


Developing  and  maintaining  smooth  and  effective  working  relationships  with 
superiors,  peers,  and  subordinates;  working  well  and  developing  effective 
relationships  with  highly  diverse  personalities. 

Being  flexible  and  open-minded  when  dealing  with  others;  listening  to  and 
considering  others’  viewpoints  and  opinions  and  altering  one’s  own  opinion 
when  appropriate;  being  open  and  accepting  of  negative  or  developmental 
feedback;  demonstrating  keen  insight  of  others’  behavior  and  tailoring  one’s 
own  behavior  to  persuade,  influence,  or  work  more  effectively  with  them. 

Helping  others  by  offering  suggestions,  teaching  them  useful  knowledge  or 
skills,  directly  performing  some  of  their  tasks,  and  providing  emotional 
support  for  their  personal  problems. 

Cooperating  with  others  by  accepting  suggestions,  informing  others  of  events 
they  should  know  about,  and  putting  team  objectives  ahead  of  personal 
interests. 

Showing  consideration,  courtesy,  and  tact  in  relations  with  others  as  well  as 
motivating  and  showing  confidence  in  them. 


Organizational  Support 


Representing  the  Representing  the  organization  to  those  not  in  the  organization;  maintaining 
Organization  to  good  organizational  image  to  the  public,  the  government,  and  others; 

the  Public  defending  and  promoting  the  organization  to  others. 


Organizational 

Commitment 


Suggesting 

Improvements 


Working  effectively  within  the  framework  of  organizational  policies, 
procedures,  and  rules;  carrying  out  orders  and  directives;  supporting  the 
organization’s  mission  and  objectives. 

Suggesting  changes  to  try  to  improve  unit  or  organizational  functioning; 
trying  to  continually  improve  the  way  things  are  being  done. 
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Conscientious  Initiative 


Persisting  to 
Reach  Goals 

Showing 

Initiative 

Self- 

Development 


Dealing  with 
Uncertain  and 
Unpredictable 
Work  Situations 


Physically 

Oriented 

Adaptability 


Persisting  with  extra  effort  to  attain  objectives,  even  under  difficult  and 
challenging  conditions;  overcoming  obstacles  to  get  the  job  done. 

Taking  the  initiative  to  do  all  that  is  necessary  to  accomplish  objectives  even 
if  they  are  not  normally  a  part  of  one’s  duties;  finding  additional  productive 
work  to  perform  when  one’s  duties  are  completed. 

Developing  knowledge  and  skills  by  taking  advantage  of  opportunities 
within  the  organization  and  outside  the  organization  through  the  use  of  one’s 
own  time  and  resources. 


Specific  Adaptive  Performance  Dimensions 

Taking  effective  action  when  necessary  without  having  to  know  the  total 
picture  or  have  all  the  facts  at  hand;  readily  and  easily  “changing  gears”  in 
response  to  unpredictable  or  unexpected  events  and  circumstances; 
effectively  adjusting  plans,  goals,  actions,  or  priorities  to  deal  with  changing 
situations;  imposing  structure  for  self  and  others  that  provides  as  much  focus 
as  possible  in  dynamic  situations;  not  needing  things  to  be  black-and-white; 
refusing  to  be  paralyzed  by  uncertainty  or  ambiguity. 

Adjusting  to  challenging  environmental  states  such  as  extreme  heat, 
humidity,  cold,  or  dirtiness;  frequently  pushing  oneself  physically  to 
complete  strenuous  or  demanding  tasks;  adjusting  weight  and  muscular 
strength  or  becoming  proficient  in  performing  physical  tasks,  as  necessary. 
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Performance  Rating  Instrument  for 
ROTC  Cadets  and  Midshipmen 

For  Research  Purposes  Only 

[Administered  Online] 


Your  ID  #  (From  the  e-mail  containing  the  link  to  this  survey: 


ID  #  of  person  you  are  rating  (from  the  list  of  participants):  _ 

Your  relationship  to  the  person  being  rated  (check  one):  □  Supervisor 

□  Peer 

□  Subordinate 


Click  NEXT»  to  continue 
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Instructions 


This  rating  form  includes  40  behavior  statements  reflecting  certain  aspects  of  the  role  of  ROTC 
cadet  or  midshipman.  You  will  be  rating  the  performance  of  a  cadet/midshipman  whose  behavior 
you  know  well  on  each  of  these  behaviors. 

Use  the  following  scale  to  indicate  the  extent  to  which  each  statement  describes  the  person  being 
rated: 

1  =  Not  at  all 

2  =  To  a  small  extent 

3  =  To  a  moderate  extent 

4  =  To  a  large  extent 

5  =  To  a  very  great  extent 
N  =  Not  observed 

Make  your  ratings  as  carefully  and  accurately  as  possible.  To  help  you  to  avoid  common  errors 
made  by  raters,  please  read  the  information  presented  on  the  next  screen  carefully. 

If  you  are  unable  to  complete  the  survey  in  one  sitting,  you  may  exit  the  survey  and  return  later 
to  finish  where  you  left  off.  Your  responses  on  each  page  will  be  saved  as  soon  as  you  advance 
by  clicking  NEXT». 

Click  NEXT»  to  continue. 
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Rating  Tips 

When  rating  the  performance  of  others,  there  are  several  types  of  common  rating  tendencies  that 
may  affect  the  accuracy  of  the  ratings.  Please  review  the  tendencies  described  below  and  keep 
this  information  in  mind  as  you  make  your  ratings. 

Halo.  This  is  the  tendency  to  give  a  person  similar  ratings  on  all  dimensions  of  performance. 

This  may  happen  if  all  ratings  are  simply  based  on  a  general  impression  of  a  person,  or  if 
performance  on  one  dimension  is  allowed  to  affect  the  ratings  on  other  dimensions.  An  example 
would  be  a  rater  who  allows  a  ratee 's  outstanding  communication  skills  to  affect  the  evaluation 
of  the  ratee ’s  personal  support  skills.  It  is  very  unlikely  that  anyone  performs  at  the  same  level 
on  all  of  the  different  dimensions  of  social  performance.  Instead,  most  people  perform  well  in 
some  areas  and  less  well  in  other  areas.  Your  ratings  should  reflect  the  strengths  and 
weaknesses  of  the  person  you  are  rating. 

Leniency/Severity.  This  is  the  tendency  to  only  give  ratings  at  one  end  of  the  scale.  An  example 
of  a  lenient  rater  would  be  someone  who  gives  only  high  ratings  because  he  or  she  wants  to 
avoid  giving  ratings  that  seem  too  “ negative .  ”  An  example  of  a  severe  rater  would  be  someone 
who  gives  only  low  ratings  to  motivate  his  or  her  subordinates  to  work  harder.  It  is  important 
that  the  ratings  are  accurate  and  reflect  the  ratee 's  performance  on  each  dimension  of  social 
performance.  This  means  that  ratings  may  be  low  on  some  dimensions,  and  high  on  others. 

Single  Incident/Recency.  This  is  the  tendency  to  be  overly  influenced  by  one  particularly 
effective  or  ineffective  example  of  a  person ’s  performance,  or  by  the  most  recent  incident 
observed.  For  example,  let ’s  say  that  last  Friday  Jane  was  exceptionally  supportive  of  another 
person  in  her  unit.  When  rating  an  item  relevant  to  the  dimension  Personal  Support,  the  rater 
remembers  that  one  incident  and  rates  Jane  a  “5.  ”  Instead,  the  rater  should  think  about  Jane ’s 
typical  performance  over  time.  The  rating  should  reflect  typical  performance  rather  than  just  one 
example  or  the  last  incident  that  can  be  remembered. 

Stereotypes.  This  is  the  tendency  to  allow  information  that  has  nothing  to  do  with  performance  to 
influence  ratings.  A  person ’s  family  background,  education,  gender,  or  previous  experience  may 
lead  a  rater  to  rate  the  person  in  certain  ways,  either  high  or  low.  An  example  is  a  rater  who 
rates  all  of  the  women  in  his  group  a  4  or  5  on  items  relevant  to  Personal  Support  because  he 
believes  all  women  are  high  on  this  dimension.  Your  ratings  should  be  based  only  on  what  you 
have  seen  the  person  do. 

Same  Level  of  Effectiveness.  This  is  the  tendency  to  give  everyone  the  same  rating.  It  is  very 
unlikely  that  all  of  the  people  you  are  rating  perform  at  the  same  level  of  effectiveness  on  a 
particular  social  performance  dimension.  As  such,  your  ratings  should  reflect  who  is  performing 
more  effectively  and  who  is  performing  less  effectively  on  each  aspect  of  social  performance. 

Although  these  tendencies  are  important  and  you  should  be  aware  of  them,  the  most  important 
thing  is  that  you  rate  each  person 's  social  performance  accurately.  Focus  on  making  accurate 
ratings,  not  on  avoiding  rating  tendencies.  If  you  think  about  what  you  are  doing  and  base  your 
ratings  on  behavior,  your  ratings  are  likely  to  be  accurate. 

Now,  click  NEXT  »  to  go  on  to  the  next  page  and  begin  making  your  ratings. 
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To  what  extent  does  each  statement  describe  the  person  you  are  rating? 


This  person: 


1 .  Makes  decisions  without  assistance  in  a  timely  manner 

2.  Finds  effective  solutions  to  problems  quickly  when  necessary 

3.  Obtains  all  relevant  information  before  making  decisions 

4.  Detects  problems  quickly  when  they  exist 

5.  Chooses  an  effective  course  of  action  based  on  available  facts 

6.  Makes  decisions  with  confidence  and  authority 

7.  Asks  team  members  for  input  when  necessary  to  solve  a  problem 

8.  Identifies  the  causes  of  problems  in  a  timely  manner 

9.  Quickly  gains  the  trust  of  others 

10.  Maintains  good  working  relationships  through  effective  communication 

1 1 .  Addresses  personal  conflicts  constructively  and  in  a  professional  manner 

12.  Develops  and  maintains  good  working  relationships  with  other  military  personnel . 

13.  Actively  participates  in  social  and  recreational  activities  arranged  for  the  unit 

14.  Treats  others  with  respect  regardless  of  their  rank,  background,  or  characteristics 

1 5.  Makes  an  effort  to  learn  and  use  others’  names . HH  2 

16.  Makes  people  feel  comfortable  about  approaching  and  talking  with  him  or  her. 

17.  Displays  a  sharp,  professional  appearance  while  in  uniform. 

18.  Exceeds  standards  when  carrying  out  orders . 

19.  Displays  commitment  to  the  unit’s  mission  and  goals 

20.  Shows  concern  for  the  success  of  the  unit/battalion . 

21 .  Behaves  in  ways  that  are  consistent  with  military  policies  and  regulations . 

22.  Works  effectively  within  the  framework  of  military  policies  and  procedures . 

23.  Performs  extra  duties  without  being  told  or  asked. 

24.  Volunteers  for  assignments  or  additional  duties. 

25.  Seeks  clarification  from  leadership  when  necessary. 

26.  Anticipates  the  needs  of  the  unit . t  i’ 

27.  Seeks  ways  to  accomplish  work  when  there  is  no  clear  solution .  1 
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28.  Initiates  projects  that  may  contribute  to  or  enhance  the  success  of  the  mission 

29.  Finds  additional  productive  work  to  perform  when  his  or  her  own  duties  are 

completed . . 


5  N 


30.  Reacts  calmly  and  confidently  to  changes  in  plans  or  unusual  circumstances . 

31 .  Responds  quickly  to  difficult  situations . 

32.  Maintains  a  flexible  approach  to  accomplishing  work . 

33.  Provides  quick,  clear,  and  effective  direction  when  faced  with  changing  situations 

or  unexpected  events . 

34.  Provides  clear  structure  to  others  in  uncertain  or  ambiguous  situations . 

35.  Works  effectively  when  situations  seem  uncertain . 

36.  Adjusts  goals  and  priorities  in  response  to  changing  situations . 

37.  Accurately  assesses  how  to  best  handle  difficult  situations  and  unexpected  events .. 

38.  Performs  excellently  in  almost  all  areas . 

39.  Exceeds  standards  and  expectations  for  performance . 

40.  Sets  an  example  of  good  performance . 


Thank  you  for  completing  our  survey  -  we  really  appreciate  your  time  and  effort! 
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Experience  Inventory 

[Administered  Online] 


Thank  you  for  taking  part  in  our  study! 

This  survey  contains  37  behaviors.  Read  each  behavior  carefully,  and  decide  which  of  the 
response  choices  best  reflects  the  frequency  with  which  you  have  engaged  in  the  behavior  in  the 
past  few  vears.  Then,  check  the  appropriate  response  choice  for  that  behavior.  Be  sure  to  mark 
one,  and  only  one,  response  choice  for  each  behavior. 

Use  the  following  scale  to  indicate  the  frequency  with  which  you  have  engaged  in  each  of  the 
behaviors: 

1  =  Never 

2  =  Once  or  Twice 

3  =  Several  Times 

4  =  Frequently  or  Routinely 

Respond  as  accurately  and  honestly  as  possible.  It  is  best  to  work  at  a  fairly  rapid  pace.  Also,  it 
is  important  to  respond  to  all  of  the  statements. 

If  you  are  unable  to  complete  the  survey  in  one  sitting,  you  may  exit  the  survey  and  return  later 
to  finish  where  you  left  off.  Your  responses  on  each  page  will  be  saved  as  soon  as  you  advance 
by  clicking  NEXT>  >. 

Please  enter  your  ID  number  from  the  information  sheet  you  were  given  at  the  in-person  data 
collection  session  or  from  your  reminder  e-mail  then  click  NEXT»  to  continue. 

Please  indicate  the  frequency  with  which  you  have  engaged  in  each  of  the  following  behaviors 
over  the  past  few  years. 

1 .  Made  decisions  independently  in  a  timely  manner. 

2.  Quickly  found  effective  solutions  to  problems. 

3.  Obtained  all  relevant  information  before  making  decisions. 

4.  Detected  problems  quickly. 

5.  Chose  an  effective  course  of  action  based  on  available  facts. 

6.  Made  decisions  with  confidence  and  authority. 

7.  Asked  team  members  for  input  when  necessary  to  solve  a  problem. 

8.  Identified  the  source  of  problems  in  a  timely  manner. 
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9.  Quickly  gained  the  trust  of  others. 

1 0.  Maintained  good  working  relationships  through  effective  communication. 

1 1 .  Addressed  personal  conflicts  constructively  and  in  a  professional  manner. 

12.  Developed  and  maintained  good  working  relationships  with  other  military  personnel. 

13.  Actively  participated  in  social  and  recreational  activities  arranged  for  the  unit. 

14.  Treated  others  with  respect  regardless  of  their  rank,  background,  or  characteristics. 

1 5.  Made  an  effort  to  learn  and  use  others’  names. 

1 6.  Made  people  feel  comfortable  about  approaching  and  talking  with  you. 

1 7.  Displayed  a  sharp,  professional  appearance  while  in  uniform. 

1 8.  Exceeded  standards  when  carrying  out  orders. 

19.  Displayed  commitment  to  the  unit’s  mission  and  goals. 

20.  Showed  concern  for  the  success  of  the  unit/battalion. 

21.  Behaved  in  ways  that  were  consistent  with  military  policies  and  regulations. 

22.  Worked  effectively  within  the  framework  of  military  policies  and  procedures. 

23.  Performed  extra  duties  without  being  told  or  asked. 

24.  Volunteered  for  assignments  or  additional  duties. 

25.  Sought  clarification  from  leadership  when  necessary. 

26.  Anticipated  the  needs  of  the  unit. 

27.  Sought  ways  to  accomplish  work  when  there  was  no  clear  solution. 

28.  Initiated  projects  that  contributed  to  or  enhanced  the  success  of  the  mission. 

29.  Found  additional  productive  work  to  perform  when  you  had  completed  your  duties. 

30.  Reacted  calmly  and  confidently  to  unusual  circumstances  or  changes  in  plans. 

3 1 .  Responded  quickly  to  difficult  situations. 

32.  Maintained  a  flexible  approach  to  accomplishing  work. 

33.  Provided  quick,  clear,  and  effective  direction  when  faced  with  changing  situations  or 
unexpected  events. 
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34.  Provided  clear  structure  to  others  in  uncertain  or  ambiguous  situations. 

35.  Worked  effectively  when  situations  seemed  uncertain. 

36.  Adjusted  goals  and  priorities  in  response  to  changing  situations. 

37.  Accurately  assessed  how  to  best  handle  difficult  situations  and  unexpected  events. 

Thank  you  for  completing  this  survey  -  we  appreciate  your  time  and  effort! 
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Appendix  D 

Revised  Action  Control  Scale 
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Action  Control  Scale 

[Administered  Online] 


Thank  you  for  taking  part  in  our  study! 

The  following  scale  is  designed  to  measure  your  action-state  orientation,  which  includes  the 
degree  to  which  you  initiate  goal-directed  actions,  detach  from  thoughts  about  interfering  goals, 
and  focus  on  a  given  task. 

Please  read  each  item  carefully,  and  then  check  the  response  that  is  more  true  of  you.  Although 
you  may  find  that  both  responses  are  applicable  to  you  or  neither  response  is  applicable  to  you, 
please  select  the  one  that  is  most  typical  of  you. 

If  you  are  unable  to  complete  the  survey  in  one  sitting,  you  may  exit  the  survey  and  return  later 
to  finish  where  you  left  off.  Your  responses  on  each  page  will  be  saved  as  soon  as  you  advance 
by  clicking  NEXT». 

Please  enter  your  ID  number  from  the  information  sheet  you  were  given  at  the  in-person  data 
collection  session  or  from  your  reminder  e-mail,  then  click  NEXT»  to  continue. 

For  each  statement,  please  check  the  response  that  is  more  true  of  you. 

1 .  When  I  know  I  must  finish  something  soon: 

a.  I  have  to  push  myself  to  get  started 

b.  I  find  it  easy  to  get  it  done  and  over  with 

2.  When  I  have  learned  a  new  and  interesting  game: 

a.  I  quickly  get  tired  of  it  and  do  something  else 

b.  I  can  really  get  into  it  for  a  long  time 

3.  If  I’ve  worked  for  weeks  on  one  project  and  then  everything  goes  completely  wrong  with 
the  project: 

a.  It  takes  me  a  long  time  to  adjust  myself  to  it 

b.  It  bothers  me  for  a  while,  but  then  I  don’t  think  about  it  anymore 

4.  When  I  don’t  have  anything  in  particular  to  do  and  am  getting  bored: 

a.  I  have  trouble  getting  up  enough  energy  to  do  anything  at  all 

b.  I  quickly  find  something  to  do 

5.  When  I  am  getting  ready  to  tackle  a  difficult  problem: 

a.  It  feels  like  I  am  facing  a  big  mountain  that  I  don’t  think  I  can  climb 

b.  I  look  for  a  way  that  the  problem  can  be  approached  in  a  suitable  manner 
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6.  If  I  had  just  bought  a  new  piece  of  equipment  (for  example,  an  iPod)  and  it  accidentally 
fell  on  the  floor  and  was  damaged  beyond  repair: 

a.  I  would  manage  to  get  over  it  quickly 

b.  It  would  take  me  a  long  time  to  get  over  it 

7.  When  I  have  to  solve  a  difficult  problem: 

a.  I  usually  don’t  have  a  problem  getting  started  on  it 

b.  I  have  trouble  sorting  things  out  in  my  head  so  that  I  can  get  down  to  working  on 
the  problem 

8.  If  I  have  to  talk  to  someone  about  something  important  and  can't  find  him  or  her 
anywhere: 

a.  I  can’t  stop  thinking  about  it,  even  while  I’m  doing  something  else 

b.  I  easily  forget  about  it  until  I  see  the  person 

9.  When  I  read  an  article  in  the  newspaper  that  interests  me: 

a.  I  usually  remain  so  interested  in  the  article  that  I  read  the  entire  article 

b.  I  still  often  skip  to  another  article  before  I’ve  completely  finished  the  first  one 

10.  When  I  am  told  that  my  work  has  been  completely  unsatisfactory: 

a.  I  don’t  let  it  bother  me  for  too  long 

b.  I  feel  paralyzed 

1 1 .  When  I  have  a  lot  of  important  things  to  do  and  they  all  must  be  done  soon: 

a.  I  often  don’t  know  where  to  begin 

b.  I  find  it  easy  to  make  a  plan  and  stick  to  it 

12.  When  one  of  my  co-workers  brings  up  an  interesting  topic  for  discussion: 

a.  It  can  easily  develop  into  a  long  conversation 

b.  I  soon  lose  interest  and  want  to  do  something  else 

13.  If  I’m  stuck  in  traffic  and  miss  an  important  appointment: 

a.  At  first,  it’s  difficult  for  me  to  start  to  do  something  else  at  all 

b.  I  quickly  forget  about  it  and  do  something  else 

14.  When  I  am  busy  working  on  an  interesting  project: 

a.  I  need  to  take  frequent  breaks  and  work  on  other  projects 

b.  I  can  keep  working  on  the  same  project  for  a  long  time 

15.  When  I  have  to  take  care  of  something  important  which  is  also  unpleasant: 

a.  I  do  it  and  get  it  over  with 

b.  It  can  take  a  while  before  I  can  bring  myself  to  it 
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1 6.  When  something  really  gets  me  down: 

a.  I  have  trouble  doing  anything  at  all 

b.  I  find  it  easy  to  distract  myself  by  doing  other  things 

1 7.  When  I  am  facing  a  big  project  that  has  to  be  done: 

a.  I  often  spend  too  long  thinking  about  where  I  should  begin 

b.  I  don’t  have  any  problems  getting  started 

1 8.  When  several  things  go  wrong  on  the  same  day: 

a.  I  usually  don’t  know  how  to  deal  with  it 

b.  I  just  keep  on  going  as  though  nothing  had  happened 

1 9.  When  I  read  something  I  find  interesting: 

a.  I  sometimes  still  want  to  put  it  down  and  do  something  else 

b.  I  will  sit  and  read  it  for  a  long  time 

20.  When  I  have  put  all  my  effort  into  doing  a  really  good  job  on  something  and  the  whole 
thing  doesn’t  work  out: 

a.  I  don’t  have  too  much  difficulty  starting  something  else 

b.  I  have  trouble  doing  anything  else  at  all 

21 .  When  I  have  an  obligation  to  do  something  that  is  boring  and  uninteresting: 

a.  I  do  it  and  get  it  over  with 

b.  It  can  take  a  while  before  I  can  bring  myself  to  do  it 

22.  When  I  am  trying  to  learn  something  new  that  I  want  to  learn: 

a.  I’ll  keep  at  it  for  a  long  time 

b.  I  often  feel  like  I  need  to  take  a  break  and  go  do  something  else  for  a  while 
Thank  you  for  taking  the  time  to  complete  this  survey  -  we  really  appreciate  your  time  and  effort! 
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Appendix  £ 
Military  Motives  Scale 
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Military  Motives  Scale 

[Administered  Online] 


Thank  you  for  taking  part  in  our  study! 

This  survey  is  designed  to  measure  some  of  your  values,  interests,  preferences,  and  attitudes 
related  to  your  involvement  with  the  military.  Please  read  the  instructions  included  in  each 
section  carefully,  and  then  make  the  appropriate  ratings. 

If  you  are  unable  to  complete  the  survey  in  one  sitting,  you  may  exit  the  survey  and  return  later 
to  finish  where  you  left  off.  Your  responses  on  each  page  will  be  saved  as  soon  as  you  advance 
by  clicking  NEXT». 

Please  enter  your  ID  number  from  the  information  sheet  you  were  given  at  the  in-person  data 
collection  session  or  from  your  reminder  e-mail  then  click  NEXT»  to  continue. 


Job  Satisfaction 

Response  scale: 

1  =  Very  dissatisfied 

2  =  Dissatisfied 

3  =  Neither  Satisfied  Nor  Dissatisfied 

4  =  Satisfied 

5  =  Very  satisfied 

How  satisfied  are  you  with  each  of  the  following  aspects  of  your  role  as  an  ROTC 
cadet/midshipman  ? 

1 .  Level  of  fulfillment/challenge 

2.  Amount  of  enjoyment  from  role 

3.  Use  of  your  skills  and  training 

4.  Level  of  competence  of  unit  staff/cadre 

5.  Level  of  competence  of  cadet/midshipman  superior 

6.  Quality  of  leadership  from  unit  staff/cadre 

7.  Quality  of  leadership  from  cadet/midshipman  superior 

8.  Amount  of  respect  from  unit  staff/cadre 

9.  Amount  of  respect  from  cadet/midshipman  superior 
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10.  Opportunity  to  select  duties  or  training  of  your  choice 

1 1 .  Assignments  to  duties  offering  technical  or  professional  development 

12.  Amount  of  compensation  (i.e.,  scholarships,  living  expenses,  additional  allowances) 

13.  Length  of  working  hours 

14.  Level  of  recognition  for  your  accomplishments 

15.  Level  of  fairness  in  how  your  performance  is  evaluated 

Military  Values 

Response  scale: 

0  =  Not  at  all  important 

1  =  Slightly  important 

2  =  Moderately  important 

3  =  Quite  important 

4  =  Very  important 

How  important  is  each  of  the  following  values  to  you? 

1 .  Loyalty  to  the  United  States  military 

2.  Loyalty  to  your  unit 

3.  Taking  responsibility  for  your  actions  and  decisions 

4.  Putting  what  is  good  for  your  fellow  cadets/midshipmen,  the  unit,  and  the  nation  before  your 
own  interests  (“service  before  self’) 

5.  Dedication  to  serving  the  United  States,  even  to  risking  your  life  in  its  defense 

6.  Commitment  to  working  as  a  member  of  a  team 

7.  Dedication  to  learning  your  job  and  doing  it  well 

8.  Personal  drive  to  succeed  in  your  work  and  advance 

9.  Being  honest,  open,  and  truthful 

1 0.  Being  disciplined  and  courageous  in  battle 

1 1 .  Standing  up  for  what  you  believe  is  right 

12.  Working  with  others  tactfully  and  with  military  courtesy 
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13.  Exhibiting  excellent  military  bearing  and  appearance 

1 4.  High  moral  standards  both  on-duty  and  off-duty 

15.  Building  and  maintaining  physical  fitness  and  stamina 

Organizational  Commitment 

Response  scale: 

1  =  Strongly  Disagree 

2  =  Disagree 

3  =  Neither  Agree  nor  Disagree 

4  =  Agree 

5  =  Strongly  Agree 

Please  indicate  the  extent  to  which  you  agree  or  disagree  with  each  of  the  following  statements: 

1 .  I  feel  like  I  fit  in  with  my  peers  in  the  ROTC. 

2.  The  ROTC  has  a  great  deal  of  personal  meaning  for  me. 

3.  I  feel  a  strong  sense  of  belonging  to  the  ROTC. 

4.  I  feel  emotionally  attached  to  the  military. 

5.  It  would  be  too  costly  for  me  to  leave  the  ROTC  in  the  near  future. 

6.  I  am  afraid  of  what  might  happen  if  I  quit  the  ROTC. 

7.  Too  much  in  my  life  would  be  interrupted  if  I  decided  to  leave  the  ROTC. 

8.  One  of  the  problems  of  leaving  the  ROTC  would  be  the  lack  of  available  alternatives. 
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Interests 


Response  scale: 

1  - 1  would  dislike  this  situation  very  much 

2=1  would  dislike  this  situation 

3=1  would  neither  like  nor  dislike  this  situation 

4=1  would  like  this  situation 

5=1  would  like  this  situation  very  much 

For  each  of  the  following  behaviors,  imagine  you  are  in  a  situation  where  you  have  to  engage  in 
the  behavior.  To  what  extent  would  you  enjoy  that  situation? 

1 .  Make  a  decision  without  assistance  based  on  available  facts. 

2.  Find  an  effective  solution  to  a  problem. 

3.  Obtain  relevant  information  before  making  a  decision. 

4.  Detect  problems  when  they  exist. 

5.  Ask  team  members  for  input  to  solve  a  problem. 

6.  Identify  the  cause  of  a  problem. 

7.  Gain  the  trust  of  others. 

8.  Maintain  good  working  relationships  through  effective  communication. 

9.  Address  personal  conflicts  with  others. 

10.  Develop  and  maintain  good  working  relationships  with  other  militaiy  personnel. 

1 1 .  Participate  in  social  and  recreational  activities  arranged  for  the  unit. 

12.  Treat  others  with  respect  regardless  of  their  rank,  background,  or  characteristics. 

13.  Learn  and  use  others’  names. 

14.  Make  people  feel  comfortable  about  approaching  and  talking  with  you. 

15.  Display  a  sharp,  professional  appearance  while  in  uniform. 

1 6.  Exceed  standards  when  carrying  out  orders. 

1 7.  Display  commitment  to  the  unit’s  mission  and  goals. 

1 8.  Show  concern  for  the  success  of  the  unit. 
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19.  Follow  military  policies  and  regulations. 

20.  Perform  an  extra  duty  without  being  told  or  asked. 

21 .  Volunteer  for  an  assignment  or  additional  duties. 

22.  Seek  clarification  from  leadership. 

23.  Anticipate  the  needs  of  the  unit. 

24.  Seek  a  way  to  accomplish  work  when  there  is  no  clear  solution. 

25.  Initiate  a  project  that  may  contribute  to  or  enhance  the  success  of  a  mission. 

26.  Find  additional  productive  work  to  perform  when  your  own  duties  are  completed. 

27.  React  to  changes  in  plans,  unusual  circumstances,  or  difficult  situations. 

28.  Maintain  a  flexible  approach  to  accomplishing  work. 

29.  Provide  direction  to  others  when  faced  with  a  changing  situation  or  an  unexpected  event. 

30.  Provide  structure  to  others  in  an  uncertain  or  ambiguous  situation. 

3 1 .  Work  in  a  situation  that  seems  uncertain. 

32.  Adjust  goals  and  priorities  in  response  to  a  changing  situation. 

33.  Assess  how  to  best  handle  a  difficult  situation  or  an  unexpected  event. 

Thank  you  for  completing  our  survey  -  we  really  appreciate  your  time  and  effort! 
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Citizenship  Motives  Scale 
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Citizenship  Motives  Scale 


Identification  Number 


Citizenship  Motives  Scale 


Instructions 

This  survey  describes  three  different  dimensions  of  being  a  good  “organizational  citizen.”  These 
are  behaviors  that  do  not  directly  contribute  to  getting  the  core  technical  work  of  the  organization 
done,  but  they  contribute  to  the  effective  functioning  of  the  organization  when  people  elect  to 
perform  them. 

Following  the  definition  of  each  type  of  citizenship  behavior,  there  are  a  number  of  statements 
describing  possible  reasons  why  a  person  would  engage  in  that  behavior.  We  would  like  you  to 
rate  how  important  each  statement  is  to  your  decision  to  engage  in  the  type  of  behavior 
described,  using  the  following  rating  scale: 

1  =  Not  at  all  important 

2  =  Slightly  important 

3  =  Moderately  important 

4  =  Quite  important 

5  =  Very  important 


To  complete  this  survey: 

•  Read  the  definition  of  the  first  dimension  of  being  a  good  organizational  citizen 
and  review  the  sample  behaviors  carefully. 

•  Keeping  the  definition  and  sample  behaviors  in  mind,  rate  how  important  each 
statement  is  to  your  decision  to  perform  similar  behaviors.  Circle  the  number  to 
the  right  of  each  statement  that  reflects  your  rating. 

•  Repeat  these  two  steps  for  the  second  and  third  dimensions  of  being  a  good 
organizational  citizen. 
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Please  review  the  following  definition  carefully  before  making  your  ratings. 

MAINTAINING  GOOD  WORKING  RELATIONSHIPS 


Developing  and  maintaining  smooth  and  effective  working  relationships  with  superiors,  peers,  and 
subordinates;  working  well  and  developing  effective  relationships  with  highly  diverse  personalities. 

Example  Behaviors: 

•  Develops  and  maintains  good  working  relationships  with  other  military  personnel. 

•  Treats  others  with  respect  regardless  of  their  rank,  background,  or  characteristics. 

•  Makes  people  feel  comfortable  about  approaching  and  talking  with  him  or  her. 

•  Quickly  gains  the  trust  of  others. 

•  Maintains  good  working  relationships  through  effective  communication. 

•  Addresses  personal  conflicts  constructively  and  in  a  professional  manner. 

•  Actively  participates  in  social  and  recreational  activities  arranged  for  the  unit. 

•  Makes  an  effort  to  learn  and  use  others’  names. 


How  important  is  each  of  the  following  motive  statements  in  your  decision  to  engage  in 
behaviors  that  maintain  good  working  relationships? 


Very  Important 


Quite  Important 


Slightly  Inr— * —  mm 


Moderately  Important 

T  .y v.  '  ■.  •  -J'  .  ■■■  '• 


3.  Because  I  want  to  be  fully  involved  in  the  organization. 

4.  Because  I  feel  pride  in  the  organization. 

5.  Because  the  organization  values  my  work. 

6.  Because  I  have  a  genuine  interest  in  my  work. 

7.  Because  I  want  to  be  a  well-informed  cadet/midshipman. 

8.  To  keep  up  with  the  latest  developments  in  the  organization. 

9.  Because  the  organization  treats  me  fairly. 

10.  Because  I  am  committed  to  the  organization. 

11.  Because  I  feel  it  is  important  to  help  those  in  need. 

12.  Because  I  believe  in  being  courteous  to  others. 

13.  Because  I  am  concerned  about  other  people’s  feelings. 

14.  Because  I  want  to  help  my  fellow  cadets/midshipmen  in  any  way  I  can. 


2.  Because  I  care  what  happens  to  the  organization. 


1 .  Because  I  want  to  understand  how  the  organization  works. 
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15.  Because  it  is  easy  for  me  to  be  helpful. 

16.  Because  I  like  interacting  with  my  fellow  cadets/midshipmen. 

17.  To  have  fun  with  my  fellow  cadets/midshipmen. 

18.  To  get  to  know  my  fellow  cadets/midshipmen  better. 

19.  To  be  friendly  with  others. 

20.  Because  I  can  put  myself  in  other  people’s  shoes. 

21 .  To  avoid  looking  bad  in  front  of  others. 

22.  To  avoid  looking  lazy. 

23.  To  look  better  than  my  fellow  cadets/midshipmen. 

24.  To  avoid  a  reprimand  from  my  superior. 

25.  Because  I  don’t  want  to  appear  irresponsible. 

26.  To  look  like  I  am  busy. 

27.  To  stay  out  of  trouble. 

28.  Because  rewards  are  important  to  me. 

29.  To  impress  my  fellow  cadets/midshipmen. 


HnnMHRBHMSHHSKHRHEfr 

Quite  Important 

■ 


Moderately  Important 
Slightly  Important 


NotatalHmportant 
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5 
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Please  review  the  following  definition  carefully  before  making  your  ratings. 


ORGANIZATIONAL  COMMITMENT 


Working  effectively  within  the  framework  of  organizational  policies,  procedures,  and  rules;  carrying  out 
orders  and  directives;  supporting  the  organization’s  mission  and  objectives. 

Example  Behaviors: 

•  Displays  a  sharp,  professional  appearance  while  in  uniform. 

•  Exceeds  standards  when  carrying  out  orders. 

•  Displays  commitment  to  the  unit’s  mission  and  goals. 

•  Shows  concern  for  the  success  of  the  unit/battalion. 

•  Behaves  in  ways  that  are  consistent  with  military  policies  and  regulations. 

•  Works  effectively  within  the  framework  of  military  policies  and  procedures. 


How  important  is  each  of  the  following  motive  statements  in  your  decision  to  engage  in 
organizational  commitment  behaviors? 


30.  Because  I  want  to  understand  how  the  organization  works. 

3 1 .  Because  I  care  what  happens  to  the  organization. 

32.  Because  I  want  to  be  fully  involved  in  the  organization. 

33.  Because  I  feel  pride  in  the  organization. 

34.  Because  the  organization  values  my  work. 

35.  Because  I  have  a  genuine  interest  in  my  work. 

36.  Because  I  want  to  be  a  well-informed  cadet/midshipman. 

37.  To  keep  up  with  the  latest  developments  in  the  organization. 

38.  Because  the  organization  treats  me  fairly. 

39.  Because  I  am  committed  to  the  organization. 

40.  Because  I  feel  it  is  important  to  help  those  in  need. 

41 .  Because  I  believe  in  being  courteous  to  others. 

42.  Because  I  am  concerned  about  other  people’s  feelings. 
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43.  Because  I  want  to  help  my  fellow  cadets/midshipmen  in  any  way  I  can. 

44.  Because  it  is  easy  for  me  to  be  helpful. 

45.  Because  I  like  interacting  with  my  fellow  cadets/midshipmen. 

46.  To  have  fun  with  my  fellow  cadets/midshipmen. 

47.  To  get  to  know  my  fellow  cadets/midshipmen  better. 

48.  To  be  friendly  with  others. 

49.  Because  I  can  put  myself  in  other  people’s  shoes. 

50.  To  avoid  looking  bad  in  front  of  others. 

51.  To  avoid  looking  lazy. 

52.  To  look  better  than  my  fellow  cadets/midshipmen. 

53.  To  avoid  a  reprimand  from  my  superior. 

54.  Because  I  don’t  want  to  appear  irresponsible. 

55.  To  look  like  I  am  busy. 

56.  To  stay  out  of  trouble. 

57.  Because  rewards  are  important  to  me. 

58.  To  impress  my  fellow  cadets/midshipmen. 


Very  Important 
Quite  Important 
Moderately  Important 
Slightly  Important 
Not  at  all  Important 
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Please  review  the  following  definition  carefully  before  making  your  ratings. 


SHOWING  INITIATIVE 


Taking  the  initiative  to  do  all  that  is  necessary  to  accomplish  objectives  even  if  they  are  not  normally  a 
part  of  one’s  duties;  finding  additional  productive  work  to  perform  when  one’s  duties  are  completed. 

Example  Behaviors: 

•  Performs  extra  duties  without  being  told  or  asked. 

•  Volunteers  for  assignments  or  additional  duties. 

•  Seeks  clarification  from  leadership  when  necessary. 

•  Anticipates  the  needs  of  the  unit. 

•  Seeks  ways  to  accomplish  work  when  there  is  no  clear  solution. 

•  Initiates  projects  that  may  contribute  to  or  enhance  the  success  of  the  mission. 

•  Finds  additional  productive  work  to  perform  when  his  or  her  own  duties  are  completed. 

How  important  is  each  of  the  following  motive  statements  in  your  decision  to  engage  in 
behaviors  that  show  initiative? 


vgx  'Aigf  '  ''  v'  * 

Very  Important 


Quite  Important 


Slightly  Important 


59.  Because  I  want  to  understand  how  the  organization  works. 

60.  Because  I  care  what  happens  to  the  organization. 

61 .  Because  I  want  to  be  fully  involved  in  the  organization. 

62.  Because  I  feel  pride  in  the  organization. 

63.  Because  the  organization  values  my  work. 

64.  Because  I  have  a  genuine  interest  in  my  work. 

65.  Because  I  want  to  be  a  well-informed  cadet/midshipman. 

66.  To  keep  up  with  the  latest  developments  in  the  organization. 

67.  Because  the  organization  treats  me  fairly. 

68.  Because  I  am  committed  to  the  organization. 

69.  Because  I  feel  it  is  important  to  help  those  in  need. 

70.  Because  I  believe  in  being  courteous  to  others. 

71.  Because  I  am  concerned  about  other  people’s  feelings. 
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Very  Important 


Quite  Important 


72.  Because  I  want  to  help  my  fellow  cadets/midshipmen  in  any  way  I  can. 

73.  Because  it  is  easy  for  me  to  be  helpful. 

74.  Because  I  like  interacting  with  my  fellow  cadets/midshipmen. 

75.  To  have  fun  with  my  fellow  cadets/midshipmen. 

76.  To  get  to  know  my  fellow  cadets/midshipmen  better. 

77.  To  be  friendly  with  others. 

78.  Because  I  can  put  myself  in  other  people’s  shoes. 

79.  To  avoid  looking  bad  in  front  of  others. 

80.  To  avoid  looking  lazy. 

81.  To  look  better  than  my  fellow  cadets/midshipmen. 

82.  To  avoid  a  reprimand  from  my  superior. 

83.  Because  I  don’t  want  to  appear  irresponsible. 

84.  To  look  like  I  am  busy. 

85.  To  stay  out  of  trouble. 

86.  Because  rewards  are  important  to  me. 

87.  To  impress  my  fellow  cadets/midshipmen. 
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Thank  you  for  completing  our  survey  -  we  really  appreciate  your  time  and  effort! 
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Appendix  G 

Proactive  Cognitions  Scale 


Proactive  Cognitions  Scale 

[Administered  online] 


Thank  you  for  taking  part  in  our  study! 

This  survey  is  designed  to  measure  your  expectancy,  self-efficacy,  and  commitment  to  behaviors 
that  are  likely  to  result  in  beneficial  outcomes.  Expectancy  consists  of  your  perceptions  of  the 
probability  of  an  action  resulting  in  beneficial  outcomes.  Self-efficacy  consists  of  your 
perceptions  of  how  well  you  might  perform  a  given  action.  Commitment  is  your  dedication  to 
carrying  out  behaviors  that  may  yield  benefits. 

Please  read  the  instructions  included  in  each  section  carefully,  and  then  make  the  appropriate 
ratings. 

If  you  are  unable  to  complete  the  survey  in  one  sitting,  you  may  exit  the  survey  and  return  later 
to  finish  where  you  left  off.  Your  responses  on  each  page  will  be  saved  as  soon  as  you  advance 
by  clicking  NEXT» . 

Please  enter  your  ID  number  from  the  information  sheet  you  were  given  at  the  in-person  data 
collection  session  or  from  your  reminder  e-mail,  then  click  NEXT»  to  continue. 

Expectancy  -  Part  I 

This  section  contains  37  behaviors  and  a  rating  scale  for  each  behavior.  Read  each  behavior 
carefully,  and  select  the  answer  corresponding  to  your  response  choice  for  that  behavior.  Be 
sure  to  select  one,  and  only  one,  response  choice  for  each  behavior. 

For  each  of  the  following  behaviors,  first  consider  the  probability  of  achieving  one  or  more  of 
the  following  outcomes  if  the  behavior  is  performed: 

-  The  unit  would  be  more  successful. 

- 1  would  receive  personal  recognition  (e.g.,  praise,  award). 

-  My  unit  would  be  favorably  recognized. 

- 1  would  advance  in  my  career. 

-  My  peers  would  be  pleased. 

- 1  would  avoid  punishment. 

- 1  would  benefit  personally  at  a  later  time  (e.g.,  a  future  favor,  time  off). 

Next,  use  the  following  scale  to  rate  the  probability  of  achieving  one  or  more  of  the  outcomes 
above: 

1  =  Highly  Improbable 

2  =  Somewhat  Improbable 

3  =  Neither  Probable  nor  Improbable 

4  =  Somewhat  Probable 

5  =  Highly  Probable 
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Respond  as  accurately  and  honestly  as  possible.  It  is  best  to  work  at  a  fairly  rapid  pace.  Also,  it 
is  important  to  respond  to  all  of  the  statements. 

1 .  Make  decisions  independently  in  a  timely  manner. 

2.  Find  effective  solutions  to  problems  quickly. 

3.  Obtain  all  relevant  information  before  making  decisions. 

4.  Detect  problems  quickly. 

5.  Choose  an  effective  course  of  action  based  on  available  facts. 

6.  Make  decisions  with  confidence  and  authority. 

7.  Ask  team  members  for  input  when  necessary  to  solve  a  problem. 

8.  Identify  the  source  of  problems  in  a  timely  manner. 

9.  Quickly  gain  the  trust  of  others. 

10.  Maintain  good  working  relationships  through  effective  communication. 

1 1 .  Address  personal  conflicts  constructively  and  in  a  professional  manner. 

12.  Develop  and  maintain  good  working  relationships  with  other  military  personnel. 

13.  Actively  participate  in  social  and  recreational  activities  arranged  for  the  unit. 

14.  Treat  others  with  respect  regardless  of  their  rank,  background,  or  characteristics. 

15.  Make  an  effort  to  learn  and  use  others’  names. 

16.  Make  people  feel  comfortable  about  approaching  and  talking  with  you. 

1 7.  Display  a  sharp,  professional  appearance  while  in  uniform. 

1 8.  Exceed  standards  when  carrying  out  orders. 

19.  Display  commitment  to  the  unit’s  mission  and  goals. 

20.  Show  concern  for  the  success  of  the  unit/battalion. 

21.  Behave  in  ways  that  are  consistent  with  military  policies  and  regulations. 

22.  Work  effectively  within  the  framework  of  military  policies  and  procedures. 

23.  Perform  extra  duties  without  being  told  or  asked. 

24.  Volunteer  for  assignments  or  additional  duties. 
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25.  Seek  clarification  from  leadership  when  necessary. 

26.  Anticipate  the  needs  of  the  unit. 

27.  Seek  ways  to  accomplish  work  when  there  is  no  clear  solution. 

28.  Initiate  projects  that  contribute  to  or  enhance  the  success  of  the  mission. 

29.  Find  additional  work  to  perform  when  you  have  completed  your  duties. 

30.  React  calmly  and  confidently  to  changes  in  plans  or  unusual  circumstances. 

3 1 .  Respond  quickly  to  difficult  situations. 

32.  Maintain  a  flexible  approach  to  accomplishing  work. 

33.  Provide  quick,  clear,  and  effective  direction  when  faced  with  changing  situations  or 
unexpected  events. 

34.  Provide  clear  structure  to  others  in  uncertain  or  ambiguous  situations. 

35.  Work  effectively  when  situations  seem  uncertain. 

36.  Adjust  goals  and  priorities  in  response  to  changing  situations. 

37.  Accurately  assess  how  to  best  handle  difficult  situations  and  unexpected  events. 
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Self-Efficacy  -  Part  II 


This  section  contains  37  behaviors  and  a  rating  scale  for  each  behavior.  Read  each  behavior 
carefully,  and  select  the  answer  corresponding  to  your  response  choice  for  that  behavior.  Be 
sure  to  select  one,  and  only  one,  response  choice  for  each  behavior. 

For  each  of  the  following  behaviors,  rate  how  effectively  you  would  perform  the  behavior. 

Use  the  following  scale  to  indicate  how  effectively  you  would  perform  the  behavior: 

1  =  Highly  Ineffective 

2  =  Somewhat  Ineffective 

3  =  Neither  Effective  nor  Ineffective 

4  =  Somewhat  Effective 

5  =  Highly  Effective 

Respond  as  accurately  and  honestly  as  possible.  It  is  best  to  work  at  a  fairly  rapid  pace.  Also,  it 
is  important  to  respond  to  all  of  the  statements. 

1 .  Make  decisions  independently  in  a  timely  manner. 

2.  Find  effective  solutions  to  problems  quickly. 

3.  Obtain  all  relevant  information  before  making  decisions. 

4.  Detect  problems  quickly. 

5.  Choose  an  effective  course  of  action  based  on  available  facts. 

6.  Make  decisions  with  confidence  and  authority. 

7.  Ask  team  members  for  input  when  necessary  to  solve  a  problem. 

8.  Identify  the  source  of  problems  in  a  timely  manner. 

9.  Quickly  gain  the  trust  of  others. 

10.  Maintain  good  working  relationships  through  effective  communication. 

1 1 .  Address  personal  conflicts  constructively  and  in  a  professional  manner. 

12.  Develop  and  maintain  good  working  relationships  with  other  military  personnel. 

13.  Actively  participate  in  social  and  recreational  activities  arranged  for  the  unit. 

14.  Treat  others  with  respect  regardless  of  their  rank,  background,  or  characteristics. 

15.  Make  an  effort  to  learn  and  use  others’  names. 
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1 6.  Make  people  feel  comfortable  about  approaching  and  talking  with  you. 

1 7.  Display  a  sharp,  professional  appearance  while  in  uniform. 

1 8.  Exceed  standards  when  carrying  out  orders. 

19.  Display  commitment  to  the  unit’s  mission  and  goals. 

20.  Show  concern  for  the  success  of  the  unit/battalion. 

21.  Behave  in  ways  that  are  consistent  with  military  policies  and  regulations. 

22.  Work  effectively  within  the  framework  of  military  policies  and  procedures. 

23.  Perform  extra  duties  without  being  told  or  asked. 

24.  Volunteer  for  assignments  or  additional  duties. 

25.  Seek  clarification  from  leadership  when  necessary. 

26.  Anticipate  the  needs  of  the  unit. 

27.  Seek  ways  to  accomplish  work  when  there  is  no  clear  solution. 

28.  Initiate  projects  that  contribute  to  or  enhance  the  success  of  the  mission. 

29.  Find  additional  work  to  perform  when  you  have  completed  your  duties. 

30.  React  calmly  and  confidently  to  changes  in  plans  or  unusual  circumstances. 

3 1 .  Respond  quickly  to  difficult  situations. 

32.  Maintain  a  flexible  approach  to  accomplishing  work. 

33.  Provide  quick,  clear,  and  effective  direction  when  faced  with  changing  situations  or 
unexpected  events. 

34.  Provide  clear  structure  to  others  in  uncertain  or  ambiguous  situations. 

35.  Work  effectively  when  situations  seem  uncertain. 

36.  Adjust  goals  and  priorities  in  response  to  changing  situations. 

37.  Accurately  assess  how  to  best  handle  difficult  situations  and  unexpected  events. 
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Commitment  -  Part  III 


This  section  contains  37  behaviors  and  a  rating  scale  for  each  behavior.  Read  each  behavior 
carefully,  and  select  the  answer  corresponding  to  your  response  choice  for  that  behavior.  Be 
sure  to  select  one,  and  only  one,  response  choice  for  each  behavior. 

For  each  of  the  following  behaviors,  rate  how  committed  you  are  to  performing  the  behavior 
during  the  semester. 

Use  the  following  scale  to  indicate  how  committed  you  are  to  performing  the  behavior  during  the 
semester: 

1  =  Not  at  All  Committed  - 1  definitely  will  not  do  this. 

2  =  Slightly  Committed  - 1  probably  will  not  do  this. 

3  =  Somewhat  Committed  - 1  may  or  may  not  do  this. 

4  =  Committed  - 1  will  probably  do  this. 

5  =  Very  Committed  - 1  will  definitely  do  this. 

Respond  as  accurately  and  honestly  as  possible.  It  is  best  to  work  at  a  fairly  rapid  pace.  Also,  it 
is  important  to  respond  to  all  of  the  statements. 

1 .  Make  decisions  independently  in  a  timely  manner. 

2.  Find  effective  solutions  to  problems  quickly. 

3.  Obtain  all  relevant  information  before  making  decisions. 

4.  Detect  problems  quickly. 

5.  Choose  an  effective  course  of  action  based  on  available  facts. 

6.  Make  decisions  with  confidence  and  authority. 

7.  Ask  team  members  for  input  when  necessary  to  solve  a  problem. 

8.  Identify  the  source  of  problems  in  a  timely  manner. 

9.  Quickly  gain  the  trust  of  others. 

10.  Maintain  good  working  relationships  through  effective  communication. 

1 1 .  Address  personal  conflicts  constructively  and  in  a  professional  manner. 

12.  Develop  and  maintain  good  working  relationships  with  other  military  personnel. 

13.  Actively  participate  in  social  and  recreational  activities  arranged  for  the  unit. 

14.  Treat  others  with  respect  regardless  of  their  rank,  background,  or  characteristics. 
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1 5.  Make  an  effort  to  learn  and  use  others’  names. 

16.  Make  people  feel  comfortable  about  approaching  and  talking  with  you. 

1 7.  Display  a  sharp,  professional  appearance  while  in  uniform. 

1 8.  Exceed  standards  when  carrying  out  orders. 

1 9.  Display  commitment  to  the  unit’s  mission  and  goals. 

20.  Show  concern  for  the  success  of  the  unit/battalion. 

21 .  Behave  in  ways  that  are  consistent  with  military  policies  and  regulations. 

22.  Work  effectively  within  the  framework  of  military  policies  and  procedures. 

23.  Perform  extra  duties  without  being  told  or  asked. 

24.  Volunteer  for  assignments  or  additional  duties. 

25.  Seek  clarification  from  leadership  when  necessary. 

26.  Anticipate  the  needs  of  the  unit. 

27.  Seek  ways  to  accomplish  work  when  there  is  no  clear  solution. 

28.  Initiate  projects  that  contribute  to  or  enhance  the  success  of  the  mission. 

29.  Find  additional  work  to  perform  when  you  have  completed  your  duties. 

30.  React  calmly  and  confidently  to  changes  in  plans  or  unusual  circumstances. 

31.  Respond  quickly  to  difficult  situations. 

32.  Maintain  a  flexible  approach  to  accomplishing  work. 

33.  Provide  quick,  clear,  and  effective  direction  when  faced  with  changing  situations  or 
unexpected  events. 

34.  Provide  clear  structure  to  others  in  uncertain  or  ambiguous  situations. 

35.  Work  effectively  when  situations  seem  uncertain. 

36.  Adjust  goals  and  priorities  in  response  to  changing  situations. 

37.  Accurately  assess  how  to  best  handle  difficult  situations  and  unexpected  events. 
Thank  you  for  completing  our  survey  -  we  appreciate  your  time  and  effort! 
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Appendix  H 

Self-Regulation  Inventory 


Self-Regulation  Inventory 

[Administered  Online] 

• 

Instructions 

The  purpose  of  this  survey  is  to  collect  information  about  your  recent  ROTC  experiences  related 
to  the  five  performance  dimensions  of  interest  in  this  study.  More  specifically,  we  would  like  you 
to  recall  your  most  recent  behavior  related  to  each  performance  dimension  and  answer  several 
questions  about  what  you  thought  and  did  while  performing  that  behavior. 

Do  not  worry  about  recalling  times  where  your  behavior  was  particularly  effective  or  ineffective. 
Instead,  for  each  performance  dimension,  please  focus  on  remembering  the  most  recent  time  in 
which  you  encountered  a  situation  where  you  could  perform  a  relevant  behavior. 

The  survey  will  take  about  20  minutes  and  you  will  receive  $5  for  your  participation.  During  this 
study,  you  will  be  asked  to  complete  a  total  of  six  surveys  of  this  type.  If  you  participate  in  each 
survey  opportunity,  you  will  receive  a  $10  bonus.  Thus,  you  will  earn  a  total  of  $40  for 
completing  all  six  surveys. 

For  each  performance  dimension: 

•  Read  the  definition  and  review  the  sample  behaviors  carefully. 

•  Think  back  over  the  past  month.  What  was  the  most  recent  ROTC-related 
situation  when  you  performed  a  behavior  related  to  the  performance  dimension? 

•  Keeping  that  instance  in  mind,  use  the  rating  scale  to  respond  to  the  questions. 


Rating  Scale 

1  =  Strongly  Disagree 

2  =  Disagree 

3  =  Neither  Agree  Nor  Disagree 

4  =  Agree 

5  =  Strongly  Agree 


If  you  are  unable  to  complete  the  survey  in  one  sitting,  you  may  exit  the  survey  and  return  later 
to  finish  where  you  left  off.  Your  responses  on  each  page  will  be  saved  as  soon  as  you  advance 
by  clicking  NEXT». 

Please  enter  your  ID  number  from  the  information  sheet  you  were  given  at  the  in-person  data 
collection  session  or  from  your  reminder  e-mail.  Then  click  NEXT»  to  continue. 
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Please  read  the  following  definition  and  review  the  sample  behaviors. 


Decision  Making/Problem  Solving 

Making  sound  and  timely  decisions;  paying  attention  to  and  taking  into  account  all 
relevant  information  in  making  decisions;  generating  creative  alternatives  to  solve 
problems;  integrating  seemingly  unrelated  information  and  developing  effective 
solutions;  anticipating  possible  problems. 

Sample  Behaviors 

•  Makes  decisions  without  assistance  in  a  timely  manner. 

•  Finds  effective  solutions  to  problems  quickly  when  necessary. 

•  Obtains  all  relevant  information  before  making  decisions. 

•  Detects  problems  quickly  when  they  exist. 

•  Chooses  an  effective  course  of  action  based  on  available  facts. 

•  Makes  decisions  with  confidence  and  authority. 

•  Asks  team  members  for  input  when  necessary  to  solve  a  problem. 

•  Identifies  the  causes  of  problems  in  a  timely  manner. 

Thinking  back  over  the  last  month,  when  did  you  most  recently  experience  a  situation  in  which  you 
could  make  a  decision  or  solve  a  problem  related  to  ROTC?  Take  a  moment  to  think  about  that  time. 
Visualize  and  walk  yourself  through  the  experience.  What  was  the  situation  leading  up  to  the 
experience?  What  did  you  do? 

[Note:  Responses  to  the  above  questions  will  not  be  collected  from  research  participants.] 


Based  on  this  experience,  please  complete  the  following  ratings. 


During  this  experience... 

Strongly 

Disagree 

Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

Strongly 

Agree 

1 .  1  found  it  difficult  to  concentrate. 

1 

2 

3 

4 

5 

2.  1  thought  about  how  poorly  1  was  doing. 

1 

2 

3 

4 

5 

3.  1  had  to  overcome  some  bad  habits. 

1 

2 

3 

4 

5 

4.  Failure  to  manage  my  negative 
emotions  hurt  how  well  1  did. 

1 

2 

3 

4 

5 

5.  1  let  distractions  undermine  my 
performance. 

1 

2 

3 

4 

5 

6.  1  thought  about  other  things. 

1 

2 

3 

4 

5 

7.  1  imagined  how  awful  failing  would  be. 

1 

2 

3 

4 

5 
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During  this  experience... 

Strongly 

Disagree 

Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

Strongly 

Agree 

8.  My  performance  was  hurt  because  1 
just  kept  doing  the  same  thing,  even 
though  it  was  not  working  well. 

1 

2 

3 

4 

5 

9.  My  performance  suffered  because  1 
could  not  bounce  back  from 
unpleasant  emotions  1  experienced. 

1 

2 

3 

4 

5 

10.  1  felt  distracted. 

1 

2 

3 

4 

5 

11.1  became  frustrated  with  myself. 

1 

2 

3 

4 

5 

12.  My  work  routines  helped  me  perform 
well. 

1 

2 

3 

4 

5 

13.  1  could  have  done  better  if  1  had  paid 
attention  to  my  progress. 

1 

2 

3 

4 

5 

14.  My  success  was  limited  because  1 
could  not  manage  to  focus  on  what  1 
was  doing. 

1 

2 

3 

4 

5 

15.  1  became  “stressed  out"  with  worry  and 
anxiety. 

1 

2 

3 

4 

5 

16.  My  mind  wandered  to  other  things. 

1 

2 

3 

4 

5 

17.  My  habitual  way  of  doing  things 
interfered  with  my  performance. 

1 

2 

3 

4 

5 
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Please  read  the  following  definition  and  review  the  sample  behaviors. 


Maintaining  Good  Working  Relationships 

Developing  and  maintaining  smooth  and  effective  working  relationships  with  superiors, 
peers,  and  subordinates;  working  well  and  developing  effective  relationships  with  highly 
diverse  personalities. 

Sample  Behaviors 

•  Quickly  gains  the  trust  of  others. 

•  Maintains  good  working  relationships  through  effective  communication. 

•  Addresses  personal  conflicts  constructively  and  in  a  professional  manner. 

•  Develops  and  maintains  good  working  relationships  with  other  military  personnel. 

•  Actively  participates  in  social  and  recreational  activities  arranged  for  the  unit. 

•  Treats  others  with  respect  regardless  of  their  rank,  background,  or  characteristics. 

•  Makes  an  effort  to  learn  and  use  others’  names. 

•  Makes  people  feel  comfortable  about  approaching  and  talking  with  him  or  her. 

Thinking  back  over  the  last  month,  when  did  you  most  recently  experience  a  situation  in  which  you 
could  work  to  maintajn  good  work  relationships  related  to  ROTC?  Take  a  moment  to  think  about 
that  time.  Visualize  and  walk  yourself  through  the  experience.  What  was  the  situation  leading  up  to 
the  experience?  What  did  you  do? 

[Note:  Responses  to  the  above  questions  will  not  be  collected  from  research  participants.] 


Based  on  this  experience,  please  complete  the  following  ratings. 


During  this  experience... 

Strongly 

Disagree 

Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

Strongly 

Agree 

18.  1  found  it  difficult  to  concentrate. 

1 

2 

3 

4 

5 

19.  1  thought  about  how  poorly  1  was  doing. 

1 

2 

3 

4 

5 

20.  1  had  to  overcome  some  bad  habits. 

1 

2 

3 

4 

5 

21.  Failure  to  manage  my  negative 

1 

2 

3 

4 

5 

emotions  hurt  how  well  1  did. 

22.  1  let  distractions  undermine  my 

1 

2 

3 

4 

5 

performance. 

23.  1  thought  about  other  things. 

1 

2 

3 

4 

5 

24.  1  imagined  how  awful  failing  would  be. 

1 

2 

3 

4 

5 
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During  this  experience... 

Strongly 

Disagree 

Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

Strongly 

Agree 

25.  My  performance  was  hurt  because  1 
just  kept  doing  the  same  thing,  even 
though  it  was  not  working  well. 

1 

2 

3 

4 

5 

26.  My  performance  suffered  because  1 
could  not  bounce  back  from 
unpleasant  emotions  1  experienced. 

1 

2 

3 

4 

5 

27.  1  felt  distracted. 

1 

2 

3 

4 

5 

28.  1  became  frustrated  with  myself. 

1 

2 

3 

4 

5 

29.  My  work  routines  helped  me  perform 
well. 

1 

2 

3 

4 

5 

30.  1  could  have  done  better  if  1  had  paid 
attention  to  my  progress. 

1 

2 

3 

4 

5 

31 .  My  success  was  limited  because  1 
could  not  manage  to  focus  on  what  1 
was  doing. 

1 

2 

3 

4 

5 

32.  1  became  “stressed  out"  with  worry  and 
anxiety. 

1 

2 

3 

4 

5 

33.  My  mind  wandered  to  other  things. 

1 

2 

3 

4 

5 

34.  My  habitual  way  of  doing  things 
interfered  with  my  performance. 

1 

2 

3 

4 

5 
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Please  read  the  following  definition  and  review  the  sample  behaviors. 


Organizational  Commitment 

Working  effectively  within  the  framework  of  organizational  policies,  procedures,  and 
rules;  carrying  out  orders  and  directives;  supporting  the  organization’s  mission  and 
objectives. 

Sample  Behaviors 

•  Displays  a  sharp,  professional  appearance  while  in  uniform. 

•  Exceeds  standards  when  carrying  out  orders. 

•  Displays  commitment  to  the  unit’s  mission  and  goals. 

•  Shows  concern  for  the  success  of  the  unit/battalion. 

•  Behaves  in  ways  that  are  consistent  with  military  policies  and  regulations. 

•  Works  effectively  within  the  framework  of  military  policies  and  procedures. 

Thinking  back  over  the  last  month,  when  did  you  most  recently  experience  a  situation  in  which  you 
could  show  your  organizational  commitment  to  ROTC?  Take  a  moment  to  think  about  that  time. 
Visualize  and  walk  yourself  through  the  experience.  What  was  the  situation  leading  up  to  the 
experience?  What  did  you  do? 

[Note:  Responses  to  the  above  questions  will  not  be  collected  from  research  participants.] 


Based  on  this  experience,  please  complete  the  following  ratings. 


During  this  experience... 

35.  I  found  it  difficult  to  concentrate. 

36.  I  thought  about  how  poorly  I  was  doing. 

37.  I  had  to  overcome  some  bad  habits. 

38.  Failure  to  manage  my  negative 
emotions  hurt  how  well  I  did. 

39.  I  let  distractions  undermine  my 
performance. 

40.  I  thought  about  other  things. 

41.  I  imagined  how  awful  failing  would  be. 

42.  My  performance  was  hurt  because  I 
just  kept  doing  the  same  thing,  even 
though  it  was  not  working  well. 


Strongly 

Disagree 

Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

Strongly 

Agree 

5 

5 

5 

5 

5 

5 

5 

5 
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During  this  experience... 

Strongly 

Disagree 

Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

Strongly 

Agree 

43.  My  performance  suffered  because  1 
could  not  bounce  back  from 
unpleasant  emotions  1  experienced. 

1 

2 

3 

4 

5 

44.  1  felt  distracted. 

1 

2 

3 

4 

5 

45.  1  became  frustrated  with  myself. 

1 

2 

3 

4 

5 

46.  My  work  routines  helped  me  perform 
well. 

1 

2 

3 

4 

5 

47.  1  could  have  done  better  if  1  had  paid 
attention  to  my  progress. 

1 

2 

3 

4 

5 

48.  My  success  was  limited  because  1 
could  not  manage  to  focus  on  what  1 
was  doing. 

1 

2 

3 

4 

5 

49.  1  became  “stressed  out”  with  worry  and 
anxiety. 

1 

2 

3 

4 

5 

50.  My  mind  wandered  to  other  things. 

1 

2 

3 

4 

5 

51 .  My  habitual  way  of  doing  things 
interfered  with  my  performance. 

1 

2 

3 

4 

5 
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Please  read  the  following  definition  and  review  the  sample  behaviors. 


Showing  Initiative 

Taking  the  initiative  to  do  all  that  is  necessary  to  accomplish  objectives  even  if  they  are 
not  normally  a  part  of  one’s  duties;  finding  additional  productive  work  to  perform  when 
one’s  duties  are  completed. 

Sample  Behaviors 

•  Performs  extra  duties  without  being  told  or  asked. 

•  Volunteers  for  assignments  or  additional  duties. 

•  Seeks  clarification  from  leadership  when  necessary. 

•  Anticipates  the  needs  of  the  unit. 

•  Seeks  ways  to  accomplish  work  when  there  is  no  clear  solution. 

•  Initiates  projects  that  may  contribute  to  or  enhance  the  success  of  the  mission. 

•  Finds  additional  productive  work  to  perform  when  his  or  her  own  duties  are 
completed. 

Thinking  back  over  the  last  month,  when  did  you  most  recently  experience  a  situation  in  which  you 
could  show  initiative  related  to  ROTC?  Take  a  moment  to  think  about  that  time.  Visualize  and  walk 
yourself  through  the  experience.  What  was  the  situation  leading  up  to  the  experience?  What  did 
you  do? 

[Note:  Responses  to  the  above  questions  will  not  be  collected  from  research  participants.] 


Based  on  this  experience,  please  complete  the  following  ratings. 

Strongly  Disagree 

During  this  experience. . .  Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

Strongly 

Agree 

52.  1  found  it  difficult  to  concentrate. 

1 

2 

3 

4 

5 

53.  1  thought  about  how  poorly  1  was  doing. 

1 

2 

3 

4 

5 

54.  1  had  to  overcome  some  bad  habits. 

1 

2 

3 

4 

5 

55.  Failure  to  manage  my  negative 

1 

2 

3 

4 

5 

emotions  hurt  how  well  1  did. 

56.  1  let  distractions  undermine  my 

1 

2 

3 

4 

5 

performance. 

57.  1  thought  about  other  things. 

1 

2 

3 

4 

5 

58.  1  imagined  how  awful  failing  would  be. 

1 

2 

3 

4 

5 
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During  this  experience... 

Strongly 

Disagree 

Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

Strongly 

Agree 

59.  My  performance  was  hurt  because  1 
just  kept  doing  the  same  thing,  even 
though  it  was  not  working  well. 

1 

2 

3 

4 

5 

60.  My  performance  suffered  because  1 
could  not  bounce  back  from 
unpleasant  emotions  1  experienced. 

1 

2 

3 

4 

5 

61.  1  felt  distracted. 

1 

2 

3 

4 

5 

62.  1  became  frustrated  with  myself. 

1 

2 

3 

4 

5 

63.  My  work  routines  helped  me  perform 
well. 

1 

2 

3 

4 

5 

64.  1  could  have  done  better  if  1  had  paid 
attention  to  my  progress. 

1 

2 

3 

4 

5 

65.  My  success  was  limited  because  1 
could  not  manage  to  focus  on  what  1 
was  doing. 

1 

2 

3 

4 

5 

66.  1  became  “stressed  out”  with  worry  and 
anxiety. 

1 

2 

3 

4 

5 

67.  My  mind  wandered  to  other  things. 

1 

2 

3 

4 

5 

68.  My  habitual  way  of  doing  things 
interfered  with  my  performance. 

1 

2 

3 

4 

5 

H-10 


Please  read  the  following  definition  and  review  the  sample  behaviors. 


Adapting  to  Uncertain  or  Changing  Situations 

Readily  and  easily  changing  gears  in  response  to  unpredictable  or  unexpected  events  and 
circumstances;  effectively  adjusting  plans,  goals,  actions,  or  priorities  to  deal  with 
changing  situations;  imposing  structure  for  self  and  others  that  provides  as  much  focus  as 
possible  in  dynamic  situations;  not  needing  things  to  be  black  and  white;  refusing  to  be 
paralyzed  by  uncertainty  or  ambiguity. 

Sample  Behaviors 

•  Reacts  calmly  and  confidently  to  changes  in  plans  or  unusual  circumstances. 

•  Responds  quickly  to  difficult  situations. 

•  Maintains  a  flexible  approach  to  accomplishing  work. 

•  Provides  quick,  clear,  and  effective  direction  when  faced  with  changing  situations 
or  unexpected  events. 

•  Provides  clear  structure  to  others  in  uncertain  or  ambiguous  situations. 

•  Works  effectively  when  situations  seem  uncertain. 

•  Adjusts  goals  and  priorities  in  response  to  changing  situations. 

•  Accurately  assesses  how  to  best  handle  difficult  situations  and  unexpected  events. 

Thinking  back  over  the  last  month,  when  did  you  most  recently  experience  a  situation  in  which  you 
could  adapt  to  uncertain  or  changing  situations  related  to  ROTC?  Take  a  moment  to  think  about 
that  time.  Visualize  and  walk  yourself  through  the  experience.  What  was  the  situation  leading  up  to 
the  experience?  What  did  you  do? 

[Note:  Responses  to  the  above  questions  will  not  be  collected  from  research  participants.] 


Based  on  this  experience,  please  complete  the  following  ratings. 


During  this  experience... 

Strongly 

Disagree 

Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

Strongly 

Agree 

69.  1  found  it  difficult  to  concentrate. 

1 

2 

3 

4 

5 

70.  1  thought  about  how  poorly  1  was  doing. 

1 

2 

3 

4 

5 

71 .  1  had  to  overcome  some  bad  habits. 

1 

2 

3 

4 

5 

72.  Failure  to  manage  my  negative 

1 

2 

3 

4 

5 

emotions  hurt  how  well  1  did. 

73.  1  let  distractions  undermine  my 

1 

2 

3 

4 

5 

performance. 

74.  1  thought  about  other  things. 

1 

2 

3 

4 

5 
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During  this  experience... 

Strongly 

Disagree 

Disagree 

Neither 
Agree  nor 
Disagree 

Agree 

Strongly 

Agree 

75.  1  imagined  how  awful  failing  would  be. 

1 

2 

3 

4 

5 

76.  My  performance  was  hurt  because  1 
just  kept  doing  the  same  thing,  even 
though  it  was  not  working  well. 

1 

2 

3 

4 

5 

77.  My  performance  suffered  because  1 
could  not  bounce  back  from 
unpleasant  emotions  1  experienced. 

1 

2 

3 

4 

5 

78.  1  felt  distracted. 

1 

2 

3 

4 

5 

79.  1  became  frustrated  with  myself. 

1 

2 

3 

4 

5 

80.  My  work  routines  helped  me  perform 
well. 

1 

2 

3 

4 

5 

81 .  1  could  have  done  better  if  1  had  paid 
attention  to  my  progress. 

1 

2 

3 

4 

5 

82.  My  success  was  limited  because  1 
could  not  manage  to  focus  on  what  1 
was  doing. 

1 

2 

3 

4 

5 

83.  1  became  “stressed  out”  with  worry  and 
anxiety. 

1 

2 

3 

4 

5 

84.  My  mind  wandered  to  other  things. 

1 

2 

3 

4 

5 

85.  My  habitual  way  of  doing  things 
interfered  with  my  performance. 

1 

2 

3 

4 

5 

Thank  you  for  taking  part  in  our  study! 
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Appendix  I 
Work  Habits  Scale 
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Work  Habits  Scale 

[Administered  Online] 

In  this  survey,  you  are  presented  with  37  behaviors  that  are  relevant  to  performance  as  an 
ROTC  cadet  or  midshipman.  For  each  of  these  behaviors,  you  will  be  asked  to  indicate  how 
much  your  habits  or  routines  help  or  interfere  with  the  behavior. 

Habits  are  patterns  of  behavior  that  people  learn  over  time  that  can  either  help  or  interfere  with 
the  performance  of  work-related  behaviors.  They  are  stylistic  ways  of  dealing  with  different 
situations  that  you  may  encounter,  but  they  are  not  necessarily  the  best  or  most  effective  ways 
of  dealing  with  these  situations.  Habits  are  your  typical  way  of  handling  a  situation,  not 
necessarily  what  you  would  do  if  you  were  carefully  selecting  the  tactics  to  use  to  achieve  a 
particular  goal.  The  following  examples  illustrate  how  habits  may  help  or  hurt  with  the 
performance  of  behaviors. 

Example  1:  Consider  the  behavior  of  “getting  to  work  on  time.  ”  Chris  sets  the  alarm  each  day, 
allows  plenty  of  time  to  drive  to  work,  and  parks  in  the  same  place.  Chris’  habitual  way  of  doing 
things  facilitates  getting  to  work  on  time.  In  contrast,  Pat  does  not  use  an  alarm  clock  and 
frequently  misses  the  bus  to  work.  Pat’s  habits  interfere  with  the  behavior  “getting  to  work  on 
time.  ” 

Example  2:  A  behavior  relevant  to  performing  well  in  some  college  courses  is:  “Excelling  on 
class  presentations.  ”  Tyler  makes  a  habit  of  meeting  with  the  teaching  assistant  or  professor  to 
clarify  what  is  expected  and  has  a  system  for  prioritizing  the  importance  of  information  to 
present.  These  habits  all  help  Tyler  excel  on  class  presentations.  Alex  always  reviews  class 
notes  and  includes  all  related  material  in  presentations.  Alex  also  routinely  practices 
presentations  in  front  of  peers  before  delivering  them  in  class.  The  first  habit  ensures  that  Alex’s 
class  presentations  are  comprehensive,  but  sometimes  makes  them  too  long.  The  second  habit 
generally  helps  Alex  do  well  on  presentations. 

These  examples  show: 

•  Multiple  habits  may  influence  the  performance  of  one  behavior. 

•  Habits  are  the  typical  or  routine  way  people  approach  situations,  not  necessarily 
the  most  effective  way  of  handling  situations. 

•  Habits  may  involve  consciously  processing  information  (e.g.,  having  a  system  for 
prioritizing  the  importance  of  information),  but  are  implemented  without  much 
thought  when  a  situation  arises  or  a  behavior  is  needed. 

Please  use  the  following  rating  scale  to  describe  the  extent  to  which  your  habitual  way  of 
doing  things  helps  or  interferes  with  each  behavior.  Note  that  we  are  not  asking  you  to  rate 
how  well  you  perform  each  behavior.  Please  focus  on  how  your  habits  help  or  interfere  with 
each  behavior. 

1  =  Interferes  a  great  deal 

2  =  Interferes  a  moderate  amount 

3  =  Interferes  a  little 

4  =  Neither  interferes  nor  helps 

5  =  Helps  a  little 

6  =  Helps  a  moderate  amount 

7  =  Helps  a  great  deal 
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When  rating  a  particular  behavior,  please  consider  situations  you’ve  encountered  in  which  you 
had  the  opportunity  to  perform  the  behavior.  Do  you  have  a  habitual  way  of  approaching  those 
situations?  If  not,  then  you  would  choose  4  (Neither  interferes  nor  helps)  because  your  habits 
have  no  influence  on  this  behavior. 

If  your  habits  would  help  you  to  perform  this  behavior,  then  you  would  choose  5,  6,  or  7, 
depending  on  the  degree  to  which  your  habits  would  facilitate  that  behavior. 

If  your  habits  would  make  it  difficult  for  you  to  perform  this  behavior,  then  you  would  choose  1, 

2,  or  3,  depending  on  the  degree  to  which  your  habits  would  interfere  with  that  behavior. 

Before  being  asked  to  rate  the  behaviors,  you  will  be  prompted  to  think  about  your  habitual 
ways  of  approaching  situations  related  to  those  behaviors.  Please  take  a  few  minutes  to  let  us 
know  what  your  habits  are.  This  will  both  help  you  prepare  to  complete  the  ratings  and  provide 
valuable  information  about  the  habits  people  have  that  facilitate  or  interfere  with  performing 
certain  sets  of  behaviors. 

If  you  are  unable  to  complete  the  survey  in  one  sitting,  you  may  exit  the  survey  and  return  later 
to  finish  where  you  left  off.  Your  responses  on  each  page  will  be  saved  as  soon  as  you  advance 
by  clicking  NEXT». 

Please  enter  your  ID  number  from  the  information  sheet  you  were  given  at  the  in-person  data 
collection  session  or  from  your  reminder  e-mail  then  click  NEXT»  to  continue. 
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Please  review  the  following  definition  of  Decision  Making/Problem  Solving: 

Making  sound  and  timely  decisions;  paying  attention  to  and  taking  into  account  all  relevant 
information  in  making  decisions;  generating  creative  alternatives  to  solve  problems;  integrating 
seemingly  unrelated  information  and  developing  effective  solutions;  anticipating  possible 
problems. 

What  habits  or  routines  do  you  have  that  help  you  to  make  good  decisions  or  solve  problems 
effectively?  That  is,  what  routine  or  typical  ways  of  behaving  make  it  easier  for  you  to  make  a 
decision  or  solve  a  problem? 


What  habits  or  routines  do  you  have  that  interfere  with  your  ability  to  make  good  decisions  or 
solve  problems  effectively?  That  is,  what  routine  or  typical  ways  of  behaving  do  you  need  to 
overcome  because  they  make  it  difficult  to  make  a  decision  or  solve  a  problem? 


For  each  of  the  following  behaviors,  please  rate  the  extent  to  which  your  habitual  way  of  doing 
things  helps  or  interferes  with  performing  the  behavior: 


[Rating  scale:  1  =  Interferes  a  great  deal 

2  =  Interferes  a  moderate  amount 

3  =  Interferes  a  little 

4  =  Neither  interferes  nor  helps 

5  =  Helps  a  little 

6  =  Helps  a  moderate  amount 

7  =  Helps  a  great  deal] 


1 .  Making  decisions  without  assistance  in  a  timely  manner. 

2.  Finding  effective  solutions  to  problems  quickly  when  necessary. 

3.  Obtaining  all  relevant  information  before  making  decisions. 

4.  Detecting  problems  quickly  when  they  exist. 

5.  Choosing  an  effective  course  of  action  based  on  available  facts. 

6.  Making  decisions  with  confidence  and  authority. 

7.  Asking  team  members  for  input  when  necessary  to  solve  a  problem. 

8.  Identifying  the  causes  of  problems  in  a  timely  manner. 
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Please  review  the  following  definition  of  Maintaining  Good  Working  Relationships: 

Developing  and  maintaining  smooth  and  effective  working  relationships  with  superiors,  peers, 
and  subordinates;  working  well  and  developing  effective  relationships  with  highly  diverse 
personalities. 

What  habits  or  routines  do  you  have  that  help  you  to  maintain  good  working  relationships?  That 
is,  what  routine  or  typical  ways  of  behaving  make  it  easier  for  you  to  work  well  and  develop 
effective  relationships  with  others? 


What  habits  or  routines  do  you  have  that  interfere  with  your  ability  to  maintain  good  working 
relationships?  That  is,  what  routine  or  typical  ways  of  behaving  do  you  need  to  overcome 
because  they  make  it  difficult  to  work  well  and  develop  effective  relationships  with  others? 


For  each  of  the  following  behaviors,  please  rate  the  extent  to  which  your  habitual  way  of  doing 
things  helps  or  interferes  with  performing  the  behavior: 


[Rating  scale:  1  =  Interferes  a  great  deal 

2  =  Interferes  a  moderate  amount 

3  =  Interferes  a  little 

4  =  Neither  interferes  nor  helps 

5  =  Helps  a  little 

6  =  Helps  a  moderate  amount 

7  =  Helps  a  great  deal] 


1 .  Quickly  gaining  the  trust  of  others. 

2.  Maintaining  good  working  relationships  through  effective  communication. 

3.  Addressing  personal  conflicts  constructively  and  in  a  professional  manner. 

4.  Developing  and  maintaining  good  working  relationships  with  other  military  personnel. 

5.  Actively  participating  in  social  and  recreational  activities  arranged  for  the  unit. 

6.  Treating  others  with  respect  regardless  of  their  rank,  background,  or  characteristics. 

7.  Making  an  effort  to  learn  and  use  others’  names. 

8.  Making  people  feel  comfortable  about  approaching  and  talking  with  you. 
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Please  review  the  following  definition  of  Organizational  Commitment: 

Working  effectively  within  the  framework  of  organizational  policies,  procedures,  and  rules; 
carrying  out  orders  and  directives;  supporting  the  organization’s  mission  and  objectives. 

What  habits  or  routines  do  you  have  that  help  you  to  display  organizational  commitment?  That 
is,  what  routine  or  typical  ways  of  behaving  make  it  easier  for  you  to  show  your  commitment  to 
ROTC? 


What  habits  or  routines  do  you  have  that  interfere  with  your  ability  to  display  organizational 
commitment?  That  is,  what  routine  or  typical  ways  of  behaving  do  you  need  to  overcome 
because  they  make  it  difficult  to  show  your  commitment  to  ROTC? 


For  each  of  the  following  behaviors,  please  rate  the  extent  to  which  your  habitual  way  of  doing 
things  helps  or  interferes  with  performing  the  behavior: 

[Rating  scale:  1  =  Interferes  a  great  deal 

2  =  Interferes  a  moderate  amount 

3  =  Interferes  a  little 

4  =  Neither  interferes  nor  helps 

5  =  Helps  a  little 

6  =  Helps  a  moderate  amount 

7  =  Helps  a  great  deal] 


1.  Displaying  a  sharp,  professional  appearance  while  in  uniform. 

2.  Exceeding  standards  when  carrying  out  orders. 

3.  Displaying  commitment  to  the  unit’s  mission  and  goals. 

4.  Showing  concern  for  the  success  of  the  unit/battalion. 

5.  Behaving  in  ways  that  are  consistent  with  military  policies  and  regulations. 

6.  Working  effectively  within  the  framework  of  military  policies  and  procedures. 


1-6 


Please  review  the  following  definition  of  Showing  Initiative: 

Taking  the  initiative  to  do  all  that  is  necessary  to  accomplish  objectives  even  if  they  are  not 
normally  a  part  of  one’s  duties;  finding  additional  productive  work  to  perform  when  one’s  duties 
are  completed. 

What  habits  or  routines  do  you  have  that  help  you  to  show  initiative?  That  is,  what  routine  or 
typical  ways  of  behaving  make  it  easier  for  you  to  take  the  initiative  to  accomplish  things  beyond 
what  is  required? 


What  habits  or  routines  do  you  have  that  interfere  with  your  ability  to  show  initiative?  That  is, 
what  routine  or  typical  ways  of  behaving  do  you  need  to  overcome  because  they  make  it  difficult 
to  take  the  initiative  to  accomplish  things  beyond  what  is  required? 


For  each  of  the  following  behaviors,  please  rate  the  extent  to  which  your  habitual  way  of  doing 
things  helps  or  interferes  with  performing  the  behavior: 


[Rating  scale:  1 
2 

3 

4 

5 

6 
7 


Interferes  a  great  deal 
Interferes  a  moderate  amount 
Interferes  a  little 
Neither  interferes  nor  helps 
Helps  a  little 

Helps  a  moderate  amount 
Helps  a  great  deal] 


1 .  Performing  extra  duties  without  being  told  or  asked. 

2.  Volunteering  for  assignments  or  additional  duties. 

3.  Seeking  clarification  from  leadership  when  necessary. 

4.  Anticipating  the  needs  of  the  unit. 

5.  Seeking  ways  to  accomplish  work  when  there  is  no  clear  solution. 

6.  Initiating  projects  that  may  contribute  to  or  enhance  the  success  of  the  mission. 

7.  Finding  additional  productive  work  to  perform  when  his  or  her  own  duties  are  completed. 
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Please  review  the  following  definition  of  Adapting  to  Uncertain  or  Changing  Situations: 

Readily  and  easily  changing  gears  in  response  to  unpredictable  or  unexpected  events  and 
circumstances;  effectively  adjusting  plans,  goals,  actions,  or  priorities  to  deal  with  changing 
situations;  imposing  structure  for  self  and  others  that  provides  as  much  focus  as  possible  in 
dynamic  situations;  not  needing  things  to  be  black  and  white;  refusing  to  be  paralyzed  by 
uncertainty  or  ambiguity. 

What  habits  or  routines  do  you  have  that  help  you  to  adapt  to  uncertain  or  changing  situations? 
That  is,  what  routine  or  typical  ways  of  behaving  make  it  easier  for  you  to  adjust  quickly  to 
unexpected  circumstances? 


What  habits  or  routines  do  you  have  that  interfere  with  your  ability  to  adapt  to  changing  or 
uncertain  situations?  That  is,  what  routine  or  typical  ways  of  behaving  do  you  need  to  overcome 
because  they  make  it  difficult  to  adjust  quickly  to  unexpected  circumstances? 


For  each  of  the  following  behaviors,  please  rate  the  extent  to  which  your  habitual  way  of  doing 
things  helps  or  interferes  with  performing  the  behavior: 


[Rating  scale:  1  = 
2  = 

3  = 

4  = 

5  = 

6  = 
7  = 


Interferes  a  great  deal 
Interferes  a  moderate  amount 
Interferes  a  little 
Neither  interferes  nor  helps 
Helps  a  little 

Helps  a  moderate  amount 
Helps  a  great  deal] 


1 .  Reacting  calmly  and  confidently  to  changes  in  plans  or  unusual  circumstances. 

2.  Responding  quickly  to  difficult  situations. 

3.  Maintaining  a  flexible  approach  to  accomplishing  work. 

4.  Providing  quick,  clear,  and  effective  direction  when  faced  with  changing  situations  or 
unexpected  events. 

5.  Providing  clear  structure  to  others  in  uncertain  or  ambiguous  situations. 

6.  Working  effectively  when  situations  seem  uncertain. 

7.  Adjusting  goals  and  priorities  in  response  to  changing  situations. 

8.  Accurately  assessing  how  to  best  handle  difficult  situations  and  unexpected  events. 
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Appendix  J 

Situational  Judgment  Test 


J-l 


ROTC  Situational  Judgment  Test 


Identification  Number. 


Instructions 

You  will  be  presented  with  written  descriptions  of  situations  that  ROTC  cadets  or  midshipmen 
could  encounter,  followed  by  a  list  of  four  or  five  possible  responses  to  the  situation.  These 
situations  and  responses  were  developed  by  cadets  and  midshipmen  like  yourself. 

You  will  be  asked  to: 

1 .  Rate  the  effectiveness  of  each  of  the  responses.  That  is,  evaluate  the  extent  to  which  the 
response  reflects  an  effective  way  to  behave  in  that  situation. 

2.  Decide  which  response  is  the  most  effective  way  to  handle  the  situation. 

3.  Decide  which  response  is  the  least  effective  way  to  handle  the  situation. 

For  each  situation,  carefully  read  the  description  of  that  situation  and  the  list  of  possible 
responses.  Then,  think  about  the  effectiveness  of  each  of  the  responses  listed.  Try  to  think  about 
how  effective  each  response  is  compared  to  all  possible  ways  to  respond  to  that  situation. 

Rate  the  effectiveness  of  each  response  using  the  following  rating  scale: 

1  =  Very  Ineffective 

2  =  Ineffective 

3  =  Slightly  Ineffective 

4  =  Not  Particularly  Effective  or  Ineffective 

5  =  Slightly  Effective 

6  =  Effective 

7  =  Very  Effective 

This  rating  scale  also  appears  at  the  top  of  each  page.  Record  your  effectiveness  rating  for  each 
response  (from  1  to  7)  on  the  line  to  the  left  of  the  response. 

In  addition,  select  which  ONE  response  out  of  those  listed  is  the  MOST  effective.  Indicate 
which  response  you  think  is  most  effective  by  circling  “M”  in  the  first  column  to  the  right  of  the 
response.  Choose  only  ONE  response  you  feel  is  the  MOST  effective,  even  if  you  think  that  the 
response  choices  include  several  effective  ways  to  handle  the  situation. 

Please  also  select  which  ONE  of  the  responses  listed  is  the  LEAST  effective.  Indicate  which 
response  you  think  is  least  effective  by  circling  an  “L”  in  the  second  column  to  the  right  of  the 
response.  Choose  only  ONE  response  you  feel  is  the  LEAST  effective,  even  if  you  think  that  the 
response  choices  include  several  ineffective  ways  to  handle  the  situation. 
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The  following  is  an  example  of  an  SJT  item  that  has  been  completed  correctly: 

Example  One  of  the  cadcts/midshipmen  in  your  squad/flight  comes  from  a  very  different 
background  than  you  do.  You  find  that  you  do  not  always  agree  with  the 
opinions  he  expresses.  What  would  you  do? 


Hating 

Most 

Effective 

/.east 

Effective 

_6 

A.  Discuss  your  opinions  with  him  and  try  to  learn  from  him. 

M 

L 

_4 

B.  Make  an  effort  to  get  to  know  the  person  outside  of  ROTC 
activities. 

M 

L 

_6 

C.  Talk  with  the  person  about  your  disagreements  and  agree  to  work 
together  even  though  your  opinions  may  be  very  different  on 
important  issues. 

CD 

L 

_3 

D.  Make  sure  the  other  person  understands  your  point  of  view. 

M 

L 

_2 

E.  Do  the  best  you  can  under  the  circumstances. 

M 

CD 

These  ratings  may  not  agree  with  your  own  opinion  of  the  effectiveness  of  these  responses,  but 
this  example  shows  you  how  we  would  like  you  to  complete  the  items. 

Note  that: 

•  There  is  a  single  number  on  each  line  to  the  left  of  the  responses,  indicating  the  rating  of 
effectiveness  for  each  response. 

•  Each  number  on  the  rating  scale  can  be  used  for  more  than  one  response. 

•  Even  though  two  responses  were  “tied”  in  terms  of  the  highest  rating,  only  one  response 
was  circled  as  most  effective  (in  the  first  column  to  the  right  of  the  responses). 

•  There  should  be  only  one  response  circled  in  the  “Most  Effective”  column  and  one 
response  circled  in  the  “Least  Effective”  column. 


You  may  begin  the  test  when  you  are  instructed  to  do  so. 
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1 

2 

3 

4 

5 

6 

7 

Very 

Ineffective 

Ineffective 

Slightly 

Effective 

Not  Particularly 
Effective  or 
Ineffective 

Slightly 

Effective 

Effective 

Very 

Ineffective 

1 .  You  are  in  charge  of  a  squad  that  must  carry  out  a  mission.  The  mission  calls  for  a  specific 
knowledge  base  that  you  do  not  possess.  As  the  leader,  you  are  being  evaluated  on  your 
ability  to  carry  out  the  mission.  What  would  you  do? 


Rating 


Most  Least 

Effectin'  Effectin' 


A.  Find  somebody  with  the  proper  knowledge  base  and  put  them  in  M 
charge. 

B.  Seek  out  experts  on  the  subject  to  assist  you.  Go  to  them  with  M 

questions,  but  make  final  decisions  yourself. 

C.  Ask  knowledgeable  squad  members  to  tell  you  what  they  know.  M 

D.  Do  the  best  you  can  without  asking  for  help  from  other  squad  M 

members.  You  cannot  appear  weak  in  front  of  them. 


L 

L 

L 

L 


2.  You  are  in  charge  of  a  group  of  people.  One  of  your  higher-ranking  subordinates  has  an 
issue  with  the  way  you  are  assigning  some  of  the  tasks.  She  sends  you  an  e-mail  in  which 
she  wonders  where  she  fits  into  the  picture  and  asks  for  clarification  of  her  job.  In  a  way,  she 
is  reprimanding  you  (the  superior  officer)  for  not  making  her  role  clear  before.  Your 
commander  is  cc’ed  on  the  e-mail.  What  action  would  you  take  to  deal  with  this  situation? 


Most 

Effective 

least 

Effective 

A.  Set  up  a  meeting  between  the  subordinate,  your  commander,  and 
yourself  to  discuss  the  situation. 

M 

L 

B.  Confront  the  subordinate  and  reprimand  her  for  cc’ing  your 

commander  on  the  e-mail.  Threaten  to  transfer  her  to  another  unit. 

M 

L 

C.  Speak  to  the  commander  about  this  and  offer  your  plan  on  how  to 
handle  the  situation.  Then  talk  to  the  subordinate  about  poorly 

M 

L 

addressing  a  concern. 

D.  Answer  her  questions  and  clarify  her  job  in  an  e-mail  also  cc’ed 
to  your  commander.  Ignore  the  chain  of  command  issue. 

M 

L 
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1  2  3  4  5  6  7 

Very  Ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 

3.  You  have  noticed  a  significant  decline  in  the  performance  of  those  under  you.  Y ou  also  note 
that  this  change  occurred  when  a  new  departmental  policy  was  implemented.  How  would 
you  address  this  situation? 


Rating 

_  A.  Call  the  NCOs  or  higher  ranking  subordinates  into  your  office 

and  sternly  correct  their  behavior. 

_  B.  Talk  with  your  subordinates  to  see  why  they  are  having  trouble 

adapting  to  the  new  policy. 

C.  Inform  your  subordinates  that  regardless  of  the  new  policy, 
performance  will  remain  high  or  privileges  will  be  revoked. 

D.  Inform  those  who  instituted  the  new  policy  that  it  is  not  working. 


4.  You  are  required  to  be  in  uniform  today,  but  your  uniform  catches  on  a  nail  and  is  tom  in 
such  a  way  that  it  is  unserviceable.  What  would  you  do? 


Rating 

Most 

Effectiw 

Lrasl 

Effective 

A. 

Wear  a  coat  or  sweater  over  the  tear  to  cover  it. 

M 

L 

B. 

Wear  the  uniform  anyway. 

M 

L 

C. 

Wear  civilian  clothing  instead.  It  is  better  not  to  have  a  uniform 
than  to  have  a  bad  looking  one. 

M 

L 

D. 

Wear  the  next  most  formal,  but  still  functional  uniform  and  get 
the  other  replaced. 

M 

L 

You  are  the  leader  of  a  tactical  training  mission.  While  conducting  the  mission,  you  receive 
a  change  to  the  previous  situation.  This  change  completely  nullifies  your  plan  and  you  don’t 

have  much  time  left  to  complete  the  mission.  What  would  you  do? 

Most 

fjMSt 

Rating 

Effective 

Effective 

A. 

Talk  to  senior  members  of  your  team  and  quickly  devise  a  new 
plan  with  their  assistance. 

M 

L 

B. 

Improvise  as  you  go.  Taking  time  for  planning  at  this  stage  will 
compromise  the  mission. 

M 

L 

C. 

Develop  a  plan  to  complete  key  aspects  of  the  mission,  even  if 
that  means  some  mission  objectives  may  not  be  fulfilled. 

M 

L 

D. 

Modify  your  old  plan  to  fit  the  new  situation.  You  have  put  a  lot 
of  work  into  it. 

M 

L 

Most  Least 

Effective  Effective 

M  L 

M  L 

M  L 

M  L 
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1 

2 

3 

4 

5 

6 

7 

Very 

Ineffective 

Ineffective 

Slightly 

Effective 

Not  Particularly 
Effective  or 
Ineffective 

Slightly 

Effective 

Effective 

Very 

Ineffective 

6.  Your  CO  is  a  very  social,  extroverted  person  who  gets  to  know  his  subordinates  and  often 
jokes  around  with  them  when  appropriate.  He  has  gotten  to  know  you  and  has  begun  joking 


with  you  and  giving  you  a  hard  time  quite  frequently.  Sometimes  his  actions  seem  to 
the  line  and  are  slightly  offensive.  What  would  you  do? 

cross 

Rating 

Most 
Fiffecri  iv 

Least 

Effective 

A.  Tolerate  it  for  a  while,  but  inform  him  of  your  feelings  if  he  starts 
crossing  too  far  over  the  line. 

M 

L 

B.  Joke  back  with  him  in  the  same  manner.  Maybe  he  will  start  to 
understand  when  he  is  crossing  the  line  if  he  is  the  butt  of  the  joke 
sometimes. 

M 

L 

C.  Tell  him  respectfully  that  you  find  his  joking  offensive  at  times. 
Make  sure  there  is  a  clear  line  to  protect  both  of  you. 

M 

L 

D.  Try  to  avoid  situations  where  he  is  likely  to  want  to  joke  around. 

M 

L 

7.  You  were  transferred  to  a  new  unit  about  three  months  ago  and  you  are  very  busy  with  all  of 
the  tasks  assigned  to  you.  You  are  having  trouble  remembering  the  names  of  the  people  in 
your  unit,  leaving  them  feeling  as  though  you  really  don’t  care  about  them.  What  would  you 
do? 

Most 

Rating  Effective 


A.  Address  the  unit  and  let  them  know  that  you  have  always  had  M 

difficulty  learning  names,  but  that  it  doesn’t  mean  you  don’t  care. 

B.  Review  your  unit’s  records/interview  notes  to  try  to  remember  M 

who  is  who. 

C.  Make  a  mental  association  between  each  person’s  name  and  M 

something  about  him/her.  Try  to  use  their  names  whenever  you 

talk  to  them. 

D.  Avoid  situations  where  you  would  have  to  use  their  names.  M 


least 

Effectiw 


L 

L 

L 
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1  2  3  4  5  6  7 

Very  Ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

Ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 

8.  You  are  working  under  someone  who  you  believe  does  not  like  you  because  of  the  menial 
jobs  you  are  given.  You  feel  you  have  higher  abilities  and  want  to  show  them.  What  would 
you  do? 

Rating 

_  A. 

B. 


C. 

D. 


9.  You  have  become  very  proficient  at  your  job  so  you  find  yourself  with  a  lot  of  free  time. 
You  see  a  lot  of  other  work  that  needs  to  be  done  around  the  unit.  However,  those  around 


you  don’t  seem  to  like  it  when  others  get  in  the  way  of  their  work.  You  don’t  want  to  look 
like  you  are  doing  nothing  half  the  time  you  are  on  the  job.  What  should  you  do? 

Ruling 

Most 

Effective 

least 

Effective 

A. 

Let  your  supervisor  know  you  want  more  work  assigned. 

M 

L 

B. 

Find  ways  to  stay  busy  with  your  own  work,  even  if  it  means 
going  over  things  that  are  already  done. 

M 

L 

C. 

Make  an  announcement  saying  you  are  free  and  open  to  helping 
anyone  who  is  overburdened. 

M 

L 

D. 

Slow  your  work  down  so  you  are  still  getting  it  done  exceedingly 
well  but  with  less  down  time. 

M 

L 

Mast 

Effective 

Least 

Effective 

Ask  your  superior  why  he/she  has  a  problem  with  you.  Tell 
him/her  that  you  are  going  to  take  this  up  the  chain  of  command 
if  something  doesn’t  change. 

M 

L 

Ignore  the  problem  and  accept  that  somebody  has  to  do  the  dirty 
work.  Your  turn  to  do  more  interesting  work  will  come 
eventually. 

M 

L 

Respectfully  tell  your  supervisor  how  you  feel.  Request  more 
demanding  opportunities  in  the  future. 

M 

L 

Ask  your  peers  if  they  have  had  a  problem  with  this  supervisor 
too.  If  so,  go  to  the  supervisor  as  a  group  and  explain  how  you  all 
feel. 

M 

L 

J-8 


1 

2 

3 

4 

5 

Very 

Ineffective 

Ineffective 

Slightly 

Effective 

Not  Particularly 
Effective  or 
Ineffective 

Slightly 

Effective 

6  7 

Effective  Very 

Ineffective 


10.  You  are  in  the  middle  of  a  close  combat  situation.  Earlier  you  were  in  a  firefight  and  some 
of  your  men  were  wounded  and  one  was  killed.  You  relocated  your  men  to  regroup  and  take 
cover  from  the  enemy.  An  injured  civilian  woman  is  lying  in  the  street  screaming.  As  you 
peer  around  the  comer  she  spots  you  and  screams  for  help.  You  can’t  give  away  your 
position  because  you  have  taken  many  casualties.  You  radio  your  superior  and  tell  him 
what’s  happening.  He  tells  you  to  take  her  out.  What  would  you  do? 

t.eaxi 

Hatinu  Effective  Effectin’ 


A.  Move  to  another  location.  The  screaming  woman  will  give  away  M 
your  position  anyway. 

B.  Do  not  shoot  the  civilian.  Doing  so  would  give  away  your  M 

position.  Instead,  maintain  your  position  and  stay  quiet. 

C.  Obey  the  order.  It  was  a  direct  order  from  a  superior  officer.  M 

D.  Do  not  obey  the  order  because  it  is  unlawful.  Instead,  radio  for  M 

help  for  the  civilian  and  find  another  location. 


L 

L 

L 

L 


1 1 .  You  are  in  a  position  demanding  very  little  work.  Many  of  your  peers,  however,  are 

swamped  with  work.  Someone  asks  you  to  take  charge  of  the  execution  of  a  training  session. 
You  have  nothing  to  gain  by  doing  it  because  it  is  that  person’s  job.  What  would  you  do? 

,W*M»  l.lV* 

Hating  Effectiw  Effective 


A.  Worry  about  your  own  work.  That  person  was  selected  for  the  M 

training  session  for  a  reason  and  he/she  should  be  the  one  who 

carries  it  out. 

B.  Take  charge  of  the  planning  and  put  all  your  effort  into  it.  You  M 

feel  the  need  to  help  out  your  colleagues,  as  well  as  get  some 
experience  in  planning  a  training  session. 

C.  Take  charge  of  the  training  but  inform  the  other  person  that  they  M 
owe  you  a  favor. 

D.  Find  something  else  to  do  that  is  not  assigned  to  anyone.  They  M 

can  handle  their  work. 


L 

L 

L 

L 
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1  2  3  4  5  6  7 

Very  Ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

Ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 

12.  You  get  a  call  saying  you  have  been  reassigned  and  you  have  24  hours  to  report.  Once  you 
do,  you  discover  you’ve  been  assigned  to  a  higher  position,  and  you’ve  been  bombarded 
with  tasks  to  complete.  How  would  you  handle  this? 


Most  Least 

tffecHvr  fiffetHw 

A.  Delegate  all  responsibilities  to  subordinates  until  you  are  familiar  M  L 
with  the  position. 

_  B.  Ask  those  more  experienced  than  you  for  help.  Take  their  M  L 

recommendations  but  complete  all  work  yourself. 

_  C.  Start  working  with  your  senior  enlisted  to  prioritize  and  complete  M  L 

the  tasks. 

D.  Set  priorities  and  take  each  task  one  by  one,  even  if  they  are  ML 

unfamiliar  to  you.  Do  not  delegate  tasks  as  this  will  make  you 
appear  weak. 


13.  You  are  in  charge  of  planning  a  function  in  which  food  is  involved.  You  assign  one  of  your 
subordinates  the  responsibility  of  ordering  it.  On  the  day  of  the  event,  the  food  does  not 
arrive.  What  would  you  do? 

Most 

Rating 


Least 

tyfhrtve 


A.  Get  on  the  phone  immediately  and  order  food  you  know  will  be  M 
delivered  quickly. 

B.  Cancel  the  event  and  reschedule  for  another  time.  Ensure  the  food  M 
is  properly  ordered  for  next  time. 

C.  Call  the  caterer  to  find  out  what  the  problem  is.  If  food  cannot  be  M 
delivered  immediately,  order  something  that  can  be  delivered 

quickly. 

D.  Press  on  with  the  event.  If  the  food  does  not  arrive,  end  the  event  M 
early. 


L 

L 

L 

L 
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1 

2 

3 

4 

5 

6 

7 

Very 

Ineffective 

Ineffective 

Slightly 

Effective 

Not  Particularly 
Effective  or 
Ineffective 

Slightly 

Effective 

Effective 

Very 

Ineffective 

14.  At  your  unit’s  barracks  you  are  in  charge  of  maintenance  and  the  overall  cleanliness  of  the 
building.  You  arc  about  to  start  routine  maintenance  when  you  discover  that  two  of 

your  subordinates  are  absent  and  you  have  fewer  supplies  than  you  planned  on.  What  would 
you  do? 


Most 

EJfcrtiu’ 

Least 

Effective 

A.  Delegate  one  subordinate  to  search  for  those  who  are  missing  and 
one  to  get  more  supplies.  Begin  the  maintenance  with  the  people 
and  supplies  you  have. 

M 

L 

B.  Find  and  locate  the  missing  subordinates.  When  they  arrive,  send 
them  to  obtain  additional  supplies  and  then  begin  work. 

M 

L 

C.  Do  what  you  can  with  the  people  and  supplies  you  have. 

Discipline  the  subordinates  when  they  return. 

M 

L 

D.  Find  other  helpers  to  fill  in  for  the  missing  subordinates.  Have 
one  of  them  obtain  the  necessary  supplies  and  begin  work. 

M 

L 

1 5.  You  have  been  tasked  with  meeting  a  tough  deadline  for  a  current  project.  You  do  not  have 
all  the  facts  regarding  the  project  and  you  do  not  have  time  to  get  the  rest  of  the  information. 
What  course  of  action  would  you  take? 


Most 

Effectin' 

least 

Effective 

A.  Complete  the  task  to  the  best  of  your  ability  with  the  facts  you  do 
have,  and  turn  in  whatever  you  have  done  at  the  deadline. 

M 

L 

B.  Request  a  new  deadline  so  that  you  can  find  out  what  you  need  to 
know  to  get  the  task  done  right. 

M 

L 

C.  Make  your  best  guess  about  which  facts  are  missing  and  proceed. 

M 

L 

D.  Do  the  best  you  can  with  the  facts  you  do  have,  but  make  a  note 
to  your  superior  that  you  were  lacking  essential  facts. 

M 

L 
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1  2  3  4  5  6  7 

Very  Ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

Ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 

16.  You  need  to  finish  a  memorandum  to  your  superior  so  it  can  be  e-mailed  to  him  in  an  hour, 
as  promised.  The  memo  is  essentially  complete;  however,  you  are  waiting  on  several  pieces 
of  information  to  check  on  some  facts.  You  are  concerned  that  you  will  not  have  this 
information  by  the  deadline,  and  you  know  your  superior  will  be  very  upset  if  you  fail  to 
meet  the  deadline.  What  would  you  do? 


Hating 

Mast 

Effective 

I.MSt 

Effective 

A. 

Make  your  superior  aware  of  the  situation  and  ask  for 
instructions. 

M 

L 

B. 

Complete  the  memorandum  based  on  the  information  you  have 
and  include  a  comment  about  the  information  still  outstanding 
and  how  it  could  alter  the  recommendation. 

M 

L 

C. 

Wait  for  the  information  before  sending  the  recommendation. 

M 

L 

D. 

Explain  the  situation  to  your  superior  and  tell  him  that  the 
recommendations  are  being  sent  now  and  the  additional 
information  will  be  sent  later. 

M 

L 

E. 

Find  other  ways  to  get  the  information  before  sending  the 
recommendation. 

M 

L 

17.  You  have  been  tasked  with  an  important  project.  While  working  on  the  project,  the  situation 
changes  dramatically  so  now  you  see  it  as  less  important  than  when  it  was  first  assigned. 
You  see  more  important  work  that  needs  to  be  done  that  is  critical  to  the  unit.  What  would 
you  do? 


Haring 


Most  least 

Effective  Effective 


A.  Switch  focus  and  energy  to  the  more  important  work. 

B.  Stick  with  your  original  plan  and  finish  what  you  have  started. 

C.  Work  on  the  project  on  the  side.  Get  more  pertinent  work  done 
first. 

D.  Ask  your  superior  if  priorities  should  be  shifted.  Volunteer  to 
shift  priorities  or  delegate  the  previous  task  to  someone  else. 


M  L 

M  L 

M  L 

M  L 
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1  2  3  4  5  6  7 

Very  ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

Ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 


1 8.  One  of  your  peers  often  wants  to  discuss,  in  front  of  other  people  in  the  unit,  his  weekend 
party  stories  including  sex,  drinking,  and  drugs.  You  want  to  remain  on  good  terms  with 
him,  but  you  don’t  want  to  be  associated  with  this  kind  of  behavior.  What  would  you  do? 

Mom  l.t'OM 

Rutin#  Effectin'  Effvvtiw' 


A.  Talk  with  him  privately  and  explain  that  his  stories  are  M 

unprofessional  and  should  not  be  discussed  in  the  workplace. 

B.  Walk  away  from  him  whenever  he  begins  one  of  his  stories.  He  M 
will  get  the  hint  after  a  while. 

C.  Tell  him  that  he  is  heading  down  a  path  of  destruction  and  needs  M 
to  get  some  help.  Help  him  find  help  if  he  wants  you  to. 

D.  Don’t  make  a  big  deal  out  of  it,  but  change  the  topic  whenever  he  M 

starts  telling  an  inappropriate  story. 


L 

L 

L 

L 


19.  You  are  a  squad  leader  and  you  find  out  that  today’s  field  training  event  is  not  going  to 
happen  outside  because  of  very  cold  weather.  Instead,  training  will  be  done  inside,  but  there 
is  not  nearly  enough  room  for  everybody  to  all  train  at  once  and  gain  something  productive 
from  it.  In  fact,  by  training  inside,  you  doubt  that  anybody  will  gain  something  productive 
from  the  training.  What  would  you  do? 

Mom  least 

Rating  EffetUrr  kjfrctiw 


A.  Devise  another  training  exercise  that  can  be  accomplished  more  M 
easily  in  an  indoor  environment  and  proceed  with  that  training. 

B.  Solicit  the  advice  of  the  group.  Inform  it  that  effective  indoor  M 

training  is  unlikely,  but  still  offer  indoor  training  as  an  alternative. 

Follow  the  majority  judgment  about  whether  to  train  or  to  cancel 
training. 

C.  Permit  half  to  leave  while  keeping  the  other  half  to  train  inside.  It  M 
will  flip-flop  next  time  this  happens. 

D.  Proceed  with  the  planned  training  indoors.  Some  training  is  better  M 
than  none. 


L 

L 


L 

L 
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1 
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3 

4 

5 

Very 

Ineffective 

Ineffective 

Slightly 

Effective 

Not  Particularly 
Effective  or 
Ineffective 

Slightly 

Effective 

6  7 


Effective  Very 

Ineffective 


20.  Someone  in  your  unit  often  falls  out  on  runs,  appearing  to  give  up.  The  rest  of  the  unit  is 
getting  angry  with  him  and  he  is  feeling  stress  and  pressure  because  of  this.  You  would  like 
to  encourage  this  person  without  embarrassing  him.  What  would  you  do? 


Rating 


Most  Li*wrt 

Effectin'  Effectin' 


E.  The  next  time  he  falls  out  during  a  run,  fall  out  yourself  and 
motivate  him  to  get  back  into  formation. 

F.  Train  with  him  on  the  side.  Go  on  runs  with  him  outside  of 
formation  runs  to  help  him  get  up  to  par. 

G.  Have  him  lead  the  run  so  it  is  at  his  pace.  Encourage  the  rest  of 
the  unit  to  cheer  him  on. 

H.  Fuel  his  competitive  spirit  by  joking  around  with  him  about  his 
weak  performance  during  runs. 


M  L 

M  L 

M  L 

M  L 


21 .  You  know  most  of  the  people  in  your  unit,  but  there  is  a  new  person  who  is  unusually  quiet 
and  not  very  social.  Everyone  else  really  only  talks  to  the  new  person  when  necessary.  This 
new  person  is  then  placed  in  your  squad.  What  would  you  do? 


Most 

Effective 

I  east 
Effective 

A.  Welcome  him/her  to  your  squad  but  do  not  make  any  other  extra 
efforts.  He/she  may  want  privacy  and  would  not  feel  comfortable 
if  given  too  much  attention. 

M 

L 

B.  Meet  with  the  person  one-on-one  to  learn  more  about  him/her. 
Inquire  how  he/she  is  adjusting  to  the  unit.  Ask  if  there  is 
anything  he/she  needs  from  you. 

M 

L 

C.  Assign  someone  who  is  sociable  in  the  unit  to  take  this  person 
under  his/her  wing  and  show  him/her  the  ropes. 

M 

L 

D.  Leave  the  new  person  alone.  As  long  as  he/she  is  doing  his/her 
job  you  shouldn’t  do  anything.  Include  this  person  in  the  normal 
activities  of  the  squad,  but  nothing  more. 

M 

L 
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1  2  3  4  5  6  7 

Very  Ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

Ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 

22.  You  are  given  one  hour  to  do  a  cleaning  job  which  takes  you  and  your  squad  15  minutes. 
You  cannot  contact  a  superior  or  any  authority,  and  you  are  at  a  training  exercise.  You  know 


you  should  not  allow  sleep  or  recreation.  What  would  you  do? 

Rutinf* 

Most 

Effmht 

l.riist 

EffM ive 

A. 

Have  your  squad  review  and  train  on  other  skills  until  you  are  in 
contact  with  a  superior. 

M 

L 

B. 

Require  your  squad  to  utilize  their  time  by  doing  physical 
training. 

M 

L 

C. 

Since  no  authority  figure  is  around,  you  assign  meaningless  tasks 
to  pass  the  time.  It  looks  like  you’re  doing  something  and  you’re 
not  allowing  any  recreation. 

M 

L 

D. 

If  no  other  tasks  need  to  be  completed,  initiate  a  professional 
conversation  that  will  benefit  those  less  experienced  in  the  squad. 

M 

L 

23.  You  are  leading  a  run  and  are  about  a  mile  away  from  your  start  point.  You  intend  to  push 
your  group  hard  but  one  of  your  cadets  is  falling  out  of  formation.  You  don’t  want  to  break 
up  the  formation  but  you  also  want  to  challenge  the  cadets.  Your  plan  was  to  intensify  the 


workout,  not  to  slow  it  down.  What  would  you  do? 

Rating 

Mom 

Effertiw 

Effettive 

A.  As  the  cadet  falls  out  of  formation,  have  the  group  circle  back  to 
pick  him/her  up.  This  will  motivate  him/her  to  pick  up  the  pace. 

M 

L 

B.  Continue  to  push  all  cadets  hard.  Instruct  everyone  that  those  who 
cannot  keep  up  will  be  doing  another  run  later  that  day. 

M 

L 

C.  Allow  the  formation  to  keep  going,  but  fall  out  of  the  lead  and 
challenge  the  one  cadet  who  is  struggling. 

M 

L 

D.  Break  your  group  into  smaller  formations  so  that  you  can 
intensify  the  workout,  but  at  different  levels. 

M 

L 
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Very  Ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

Ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 

24.  The  base  commander  has  sent  out  an  email  looking  for  someone  to  oversee  and  set  up  the 
base  Christmas  party.  There  is  no  shortage  of  people  who  could  handle  this  task,  but  most 
people  don’t  want  to  take  on  extra  responsibilities  around  that  time  of  year.  What  would  you 
do? 


Rating 


Most 

Effective 

Least 

Effective 

Volunteer  and  use  it  as  an  opportunity  to  showcase  your  talent 
when  no  one  else  would. 

M 

L 

Wait  until  you  are  ordered  to  take  on  the  responsibility  then 
showcase  your  talent  with  the  opportunity. 

M 

L 

Talk  to  others  and  try  to  get  a  group  together  so  that  no  one 
person  will  have  such  a  great  duty. 

M 

L 

Make  suggestions  to  the  base  commander  for  who  you  think 
would  do  the  best  job. 

M 

L 

25.  When  wearing  your  uniform,  you  are  approached  by  a  group  of  people.  They  want  to  know 
how  you  feel  about  certain  political  situations,  and  then  they  start  to  criticize  you  for  just 
being  a  member  of  the  military.  They  are  acting  hostile  and  you  want  them  to  leave  you 
alone.  What  would  you  do? 


Most  Lea* 

Effective  Effective 


A.  Express  your  views.  If  you  give  the  impression  of  trying  to  get  M  L 

away  it  will  reflect  poorly  on  you  and  your  respective  service. 

B.  Refer  them  to  a  public  affairs  officer  to  answer  their  questions  or  M  L 

concerns. 


C.  Ignore  them  and  walk  away.  You  represent  the  military,  and  M  L 

therefore  can’t  speak  negatively  of  it. 

D.  Respond  with  no  comment.  In  uniform  you  do  not  have  a  political  M  L 

view. 


J-16 


1 

Very 

Ineffective 

2  3  4  5  6 

Ineffective  Slightly  Not  Particularly  Slightly  Effective 

Effective  Effective  or  Effective 

Ineffective 

7 

Very 

Ineffective 

26.  The  order  you  received  from  your  leadership  contains  what  you  consider  to  be  unnecessary 
steps.  You  know  how  to  accomplish  the  same  thing  in  half  the  time.  Your  leadership  will  not 
listen  to  your  requests  to  do  this  mission  differently.  What  would  you  do? 

Rating 

Mom 

Effective 

Lome 

hffertln 

A.  Complete  the  mission  your  way.  The  command  will  be  impressed 
and  will  be  more  open  to  suggestions  in  the  future. 

M 

L 

B.  Carry  out  the  order  as  given.  When  the  task  is  done  you  can 
suggest  a  better  way. 

M 

L 

C.  Go  up  the  chain  of  command  to  ask  your  supervisor’s  supervisor 
about  the  short  cut. 

M 

L 

D.  Conduct  the  mission  your  way  but  don’t  inform  the  leadership. 

M 

L 

27.  You  are  in  charge  of  inspecting  your  unit’s  equipment  before  a  mission.  You  know  there  is  a 
“proper”  way  to  do  this,  but  today  you  don’t  have  time  to  deal  with  all  of  the  formalities  that 
come  with  the  proper  way,  or  you  won’t  be  able  to  accomplish  the  mission  in  time.  What 
would  you  do? 


Rating 


Most  l, east 

Effective  Effective 


A.  Mission  accomplishment  comes  first.  Inform  the  unit  that  the 
formalities  will  be  skipped  this  time  due  to  time  constraints. 

B.  Seek  out  additional  manpower  to  complete  the  inspection 
properly  within  the  time  given. 

C.  Inform  your  supervisor  that  the  unit  is  behind  schedule. 

D.  Get  as  much  done  the  “proper”  way  as  possible  within  the  time 
allotted.  Leave  the  rest  undone. 


M  L 

M  L 

M  L 

M  L 
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Very  Ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

Ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 

28.  At  the  end  of  a  training  exercise  your  superior  asks  for  volunteers  to  assist  in  clean  up  of  the 
training  area.  Everyone  in  your  unit  is  tired  and  hungry  and  ready  to  go  home.  But  no  one 
leaves  until  the  area  is  cleaned.  What  would  you  do? 


Raring 

Mom 

Effetiive 

Least 

Effective 

A. 

Hold  everyone  back  until  it  is  all  cleaned  up. 

M 

L 

B. 

Rally  all  your  unit  members  together  and  motivate  them  to  all 
help  clean  up,  because  the  sooner  everything’s  clean,  the  sooner 

M 

L 

everyone  can  go  home. 

C. 

Assign  a  cleaning  crew  to  complete  the  task. 

M 

L 

D. 

Begin  cleaning  and  hope  that  others  follow  your  lead. 

M 

L 

>.  You’re  sent  to  a  new  command  and  when  you  arrive,  you  notice  almost  everyone  is  very 

casual  about  uniform  regulations.  When  you  wear  your  uniform  strictly  according  to 
regulations,  you  notice  that  others  tend  to  look  at  you  with  contempt.  What  would  you  do? 


Rating 

Mom 

Effective 

Least 

Effective 

A. 

Discuss  uniform  regulations  with  your  superior  to  find  out  what 
expectations  are  in  this  company. 

M 

L 

B. 

Try  to  befriend  the  others  in  the  company  to  gain  approval 
without  lowering  standards. 

M 

L 

C. 

Continue  wearing  the  uniform  properly.  Over  time,  they  will  want 
to  improve  their  own  uniforms. 

M 

L 

D. 

Inform  your  superior  of  your  peers  who  are  not  up  to  standards. 

M 

L 

30.  You  are  midway  through  an  important  project  when  your  superior  orders  several  changes.  It 

is  unclear  what  exactly  he  wants.  Further,  given  these  new  changes,  you  are  unsure  whether 

you  are  expected  to  meet  the  same  deadlines.  What  would  you  do? 

Most 

Leant 

Rating 

_ _ _  '  ,  .  . . . . .  . . . .  . . . -  ■ 

Efft'rtive 

Effective 

A. 

Seek  the  input  of  NCOs  or  other  unit  members  to  clear  up  the 
situation. 

M 

L 

B. 

Make  the  appropriate  changes  to  accomplish  the  new  objectives 
by  the  same  deadline. 

M 

L 

C. 

Ask  your  superior  for  further  clarification  and  guidance. 

M 

L 

D. 

Ask  for  clarification  on  the  time  frame  for  the  deadline. 

M 

L 

J-18 


1  2  3  4  5  6  7 

Very  Ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

Ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 


3 1 .  You  give  a  group  a  set  of  tasks  to  complete.  Some  tasks  are  done  well  but  some  are  left 
unfinished  or  are  poorly  done.  You  need  to  evaluate  each  individual’s  performance  as  well 
as  the  team  performance.  How  would  you  determine  who  did  good  work  and  who  did  not? 

Most  Least 

Rutin#  Effective  Effective 


A.  Privately  ask  someone  to  inform  you  which  individuals  were  not  M 
doing  their  fair  share  of  the  work. 

B.  Ask  the  group  to  do  peer  evaluations,  and  see  who  they  think  did  M 
a  good  or  bad  job. 

C.  Rate  the  team  as  a  whole  and  assign  the  group  grade  to  every  M 

individual. 

D.  Ask  everyone  to  show  you  what  they  did  personally.  M 


L 

L 

L 

L 


32.  You  are  given  an  order  to  cover  something  up  that  your  chain  of  command  did  wrong.  It 
wasn’t  a  big  mistake,  but  you  know  that  it  will  make  you  look  bad  if  it  comes  out.  Therefore, 
following  the  order  will  be  better  for  you  and  the  group  that  made  the  mistake.  What  would 
you  do? 

Most  least 

Rating  Effective  Effective 


A.  Refuse  to  follow  the  order  on  grounds  that  it  is  unlawful.  M 

B.  Cover  it  up  as  long  as  you  don’t  have  to  break  any  laws  or  alter  M 
records. 

C.  Cover  it  up.  Not  only  were  you  ordered  to,  but  your  superiors  will  M 
think  highly  of  you  for  covering  for  their  mistakes. 

D.  Don’t  cover  up  the  incident.  This  would  question  your  integrity.  M 
Owning  mistakes  is  more  important  than  looking  good. 


L 

L 

L 

L 


J-19 


1  2  3  4  5  6  7 

Very  Ineffective  Slightly  Not  Particularly  Slightly  Effective  Very 

Ineffective  Effective  Effective  or  Effective  Ineffective 

Ineffective 

33.  You  have  several  subordinates  under  you  and  they  are  having  trouble  getting  along  with 
each  other.  You  believe  the  dispute  is  over  something  very  insignificant,  but  they  do  not  see 
it  the  way  you  do.  Their  behavior  is  starting  to  affect  the  whole  unit.  What  would  you  do? 


Rating 

Most 

Effective 

Least 

Effectin' 

A. 

Ignore  it.  These  things  pass  with  time  and  bringing  attention  to 
the  issue  will  likely  make  it  worse. 

M 

L 

B. 

Bring  in  everyone  from  the  unit  and  have  an  open  discussion 
about  teamwork. 

M 

L 

C. 

Arrange  a  personal  meeting  with  each  of  them  to  get  all  sides  of 
the  story.  Afterwards,  tell  them  your  solution. 

M 

L 

D. 

Transfer  one  or  more  of  the  troublemakers  out  of  the  unit. 

M 

L 

34.  While  conducting  combat  training  operations,  your  superior  tells  you  to  kill  a  wounded 

enemy  so  that  your  unit  can  move  on  and  complete  the  mission.  This  order  seems  contrary  to 
normal  conduct  and  causes  you  to  not  immediately  carry  it  out.  Your  superior  orders  it  a 
second  time  and  waits  for  your  response.  What  would  you  do? 


Raring 


Most  i  east 

Effective  Effective 


A.  Consult  the  superior  in  confidence  and  discuss  the  issue  of  it 
being  a  lawful  order. 

B.  Do  not  obey  the  order,  if  you  believe  it  to  be  unlawful.  Do  your 
best  to  exit  the  area  without  killing  the  injured  enemy. 

C.  Obey  the  given  order  without  further  question. 

D.  Refuse.  It  is  against  the  laws  of  war  to  kill  the  wounded  or 
prisoners  as  they  are  now  non-combatants. 


M  .  L 

M  L 

M  L 

M  L 
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1 

2 

3 

4 

5 

6 

7 

Very 

Ineffective 

Ineffective 

Slightly 

Effective 

Not  Particularly 
Effective  or 
Ineffective 

Slightly 

Effective 

Effective 

Very 

Ineffective 

35.  You  arc  assigned  a  task  that  you  have  little  experience  with,  but  you  have  created  a  plan  to 
get  the  work  done  anyway.  There  is  someone  below  you  who  has  done  this  before  and  has 
some  different  ideas  on  how  to  do  it.  You  are  being  evaluated  on  your  leadership  skills. 

What  would  you  do? 

Most  Leant 

Run  nii  Effective  Effective 


A.  Stick  with  your  own  plan.  You  are  the  leader  and  it  is  your  M 

responsibility  to  direct  the  planning. 

B.  Ask  your  subordinate  what  he/she  would  do  differently  and  why.  M 
Modify  your  plan  if  the  reasoning  makes  sense. 

C.  Make  the  subordinate  a  joint  project  leader  with  yourself.  M 

D.  Remove  the  subordinate  from  the  decision-making  process.  You  M 
cannot  afford  to  have  your  plan  challenged  by  the  subordinate. 


L 

L 

L 

L 


36.  You  are  a  newly  commissioned  officer.  At  your  unit  you  work  closely  with  an  officer  of  the 
same  rank  as  you.  You  and  the  other  officer  have  very  different  personalities,  which  causes  a 
lot  of  conflict  between  the  two  of  you.  The  tension  in  your  working  relationship  is  clearly 


visible  to  the  entire  unit.  What  would  you  do? 

• 

Most 

least 

Ratine 

,  .  .  .  ..  ..  .  ..  .  L . ! . '  '  HI..  .Iff 

Effective 

Effective 

A. 

Ignore  the  other  officer  as  much  as  you  can. 

M 

L 

B. 

Ask  to  have  a  moderator  meet  with  both  of  you  to  try  to  get 
professional  help  with  the  issues  between  you. 

M 

L 

C. 

Do  your  best  to  portray  a  good  working  relationship  on  the 
outside  even  if  you  really  dislike  him  on  the  inside. 

M 

L 

D. 

Set  up  a  time  to  meet  with  the  other  officer  after  work.  Try  to  get 
to  know  each  other  better  to  figure  out  why  there  is  tension. 

M 

L 

Thank  you  very  much  for  completing  this  test. 
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Past  Behavior  Record 


Identification  Number 


The  purpose  of  this  booklet  is  to  capture  examples  of  behavior  related  to  the 
performance  dimensions  of  interest  in  this  study.  More  specifically,  we  would  like  you  to 
recall  and  describe  an  example  that  illustrates  your  best  behavior  related  to  each  of  five 
performance  dimensions.  To  help  structure  the  information,  we  ask  you  to  give  three 
pieces  of  information  about  each  example:  the  situation,  what  you  did  (the  behavior  or 
action),  and  the  outcomes  or  consequences  of  the  action. 

To  complete  this  booklet,  please: 

o  Read  the  target  dimension  name  and  definition  at  the  top  of  the  page. 

o  Read  the  list  of  example  behaviors  that  are  relevant  to  the  target  dimension. 

o  Write  one  ROTC-related  behavioral  example  for  each  target  performance  dimension.  Pick 
an  example  that  illustrates  your  best  behavior  related  to  the  target  dimension.  That  is,  try 
to  think  of  a  recent  example  that  shows  you  doing  your  best  on  the  target  dimension. 

■  Describe  an  ROTC-related  situation  from  your  recent  past  that  required  you  to 
exhibit  your  best  behavior  relevant  to  the  target  dimension. 

■  Describe  the  action(s)  you  took  in  response  to  the  situation. 

■  Describe  the  outcome  or  result  of  your  actions. 

o  If  you  can’t  think  of  a  ROTC-related  example,  you  should  provide  an  example  related  to 
the  target  dimension  from  another  setting. 

o  Make  sure  you  can  remember  enough  about  the  incident  that  you  can  describe  it  in 
enough  detail.  Please  write  as  little  as  possible  while  including  all  the  relevant  information. 
Remember,  we  are  only  interested  in  situation-behavior-outcome,  so  don’t  include  any 
unnecessary  information.  However,  be  sure  to  include  enough  information  that  the  reader 
will  know  exactly  what  you  did. 

o  Be  as  accurate  as  possible. 


K-2 


DECISION  MAKING/PROBLEM  SOLVING 


Making  sound  and  timely  decisions;  paying  attention  to  and  taking  into  account  all  relevant 
information  in  making  decisions;  generating  creative  alternatives  to  solve  problems;  integrating 
seemingly  unrelated  information  and  developing  effective  solutions;  anticipating  possible  problems. 

Example  Behaviors: 

•  Makes  decisions  without  assistance  in  a  timely  manner. 

•  Finds  effective  solutions  to  problems  quickly  when  necessary. 

•  Chooses  an  effective  course  of  action  based  on  available  facts. 

Please  answer  the  following  questions  as  concisely  as  possible. 

1 .  Recall  a  situation  from  your  recent  past  (preferably  ROTC-related)  that  required  you  to  make 

a  difficult  decision  or  solve  a  difficult  problem.  Describe  the  situation.  What  was  the  decision 
or  problem?  Describe  why  it  was  difficult  and/or  important  to  make  a  good  decision. 


2.  Describe  exactly  what  you  did  in  response  to  the  situation.  What  were  the  specific  steps  you 
took  to  decide  on  a  course  of  action?  What  different  things  did  you  consider?  What  information 
did  you  gather,  and  how?  How  was  your  response  particularly  effective,  innovative,  or 
resourceful? 


3.  What  was  the  outcome  or  result  of  what  you  did? 


K-3 


MAINTAINING  GOOD  WORKING  RELATIONSHIPS 


Developing  and  maintaining  smooth  and  effective  working  relationships  with  superiors, 

peers,  and  subordinates;  working  well  and  developing  effective  relationships  with  highly 

diverse  personalities. 

Example  Behaviors: 

•  Develops  and  maintains  good  working  relationships  with  other  military  personnel. 

•  Treats  others  with  respect  regardless  of  their  level,  background,  or  characteristics. 

•  Makes  people  feel  comfortable  about  approaching  and  talking  with  him  or  her. 

Please  answer  the  following  questions  as  concisely  as  possible. 

1 .  Recall  a  specific  and  challenging  situation  from  your  recent  past  (preferably  ROTC-related) 
that  required  you  to  take  action  to  develop  or  maintain  effective  working  relationships  with 
one  or  more  other  people.  Describe  this  specific  situation.  What  was  your  relation  to  the 
individual(s)  (e.g.,  peer,  superior,  subordinate)?  In  what  way  was  the  situation  difficult  or 
challenging?  How  did  you  know  that  you  had  to  do  something  in  order  to  develop  or  maintain 
the  relationship(s)? 


2.  Describe  exactly  what  you  did  in  order  to  develop  or  maintain  the  relationship(s).  Be  as 

specific  as  possible  about  what  you  did  and  said.  How  did  you  deal  with  any  challenges  in  the 
situation,  such  as  differences  with  the  other  person(s)? 


3.  What  was  the  outcome  or  result  of  what  you  did? 


K-4 


ORGANIZATIONAL  COMMITMENT 


Working  effectively  within  the  framework  of  organizational  policies,  procedures,  and  rules; 

carrying  out  orders  and  directives;  supporting  the  organization’s  mission  and  objectives. 

Example  Behaviors: 

•  Works  effectively  within  the  framework  of  military  policies  and  procedures. 

•  Displays  a  sharp,  professional  appearance  while  in  uniform. 

•  Shows  concern  for  the  success  of  the  unit/battalion. 

•  Carries  out  orders. 

Please  answer  the  following  questions  as  concisely  as  possible. 

1 .  Recall  a  specific  situation  from  your  recent  past  (preferably  ROTC-related)  that  required  you 
to  demonstrate  your  commitment  to  the  organization.  Describe  this  specific  situation.  What 
was  the  organization?  What  were  the  rules,  procedures,  orders,  or  objectives  that  you  had  to 
follow  and/or  support?  Why  was  it  important  to  do  so?  Was  it  difficult  to  demonstrate 
organizational  commitment  in  this  situation,  and  if  so,  why? 


2.  Describe  exactly  what  you  did  in  order  to  demonstrate  organizational  commitment  in  this 
situation.  How  did  you  address  any  difficulty  and/or  conflict  posed  by  the  situation? 


3.  What  was  the  outcome  or  result  of  what  you  did? 


K-5 


SHOWING  INITIATIVE 


Taking  the  initiative  to  do  all  that  is  necessary  to  accomplish  objectives  even  if  they  are  not 

normally  a  part  of  one’s  duties;  finding  additional  productive  work  to  perform  when  one’s  duties  are 

completed. 

Example  Behaviors: 

•  Volunteers  for  assignments  or  additional  duties. 

•  Seeks  ways  to  accomplish  work  when  there  is  no  clear  solution. 

•  Finds  additional  productive  work  to  perform  when  his  or  her  own  duties  are  completed. 

Please  answer  the  following  questions  as  concisely  as  possible. 

1 .  Recall  a  specific  situation  from  your  recent  past  (preferably  ROTC-related)  that  gave  you  an 
opportunity  to  show  initiative.  Describe  the  situation.  What  was  the  opportunity?  How  strong 
was  the  pressure  from  others,  if  any,  to  take  on  this  additional  work?  To  what  extent  did  you 
have  a  choice  about  taking  on  this  additional  responsibility? 


2.  Describe  exactly  what  you  did  to  show  initiative  in  this  situation.  Include  in  your  description 
what  your  initial  reaction  to  the  situation  was  and  what  efforts  you  made  to  meet  your 
objective(s). 


3.  What  was  the  outcome  or  result  of  what  you  did? 


K-6 


ADAPTING  TO  CHANGING  OR  UNCERTAIN  SITUATIONS  [ 


Easily  changing  gears  in  response  to  unpredictable  or  unexpected  events  and  circumstances; 

effectively  adjusting  plans,  goals,  actions,  or  priorities  to  deal  with  changing  situations;  imposing 

structure  for  self  and  others  that  provides  as  much  focus  as  possible  in  dynamic  situations;  not 

needing  things  to  be  black  and  white;  refusing  to  be  paralyzed  by  uncertainty  or  ambiguity. 

Example  Behaviors: 

•  Reacts  calmly  and  confidently  to  changes  in  plans  or  unusual  circumstances. 

•  Provides  quick,  clear,  and  effective  direction  when  faced  with  changing  situations  or 
unexpected  events. 

•  Provides  clear  structure  to  others  in  uncertain  or  ambiguous  situations. 

Please  answer  the  following  questions  as  concisely  as  possible. 

1 .  Recall  a  specific  situation  from  your  recent  past  (preferably  ROTC-related)  that  required  you 
to  adapt  to  a  changing  or  uncertain  situation.  Describe  the  situation.  What  was  the 
unexpected  event  or  situation  that  was  changing?  How  did  the  situation  change  or  how  was  it 
uncertain? 


2.  Describe  exactly  what  you  did  in  order  to  adapt  to  this  situation.  How  quickly  did  you  take 
action?  Was  this  your  decision  or  did  someone  ask  you  to  do  this?  Did  you  direct  others,  and 
if  so,  how? 


3.  What  was  the  outcome  or  result  of  what  you  did? 


K-7 


K-8 
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Conventional  Past  Behavior  Record  (PBR)  Evaluation  Guide 
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Past  Behavior  Record: 
Evaluation  Guide 


VERSION:  B  18-Sep-06 


Prepared  for: 

U.S.  Army  Research  Institute 


Candidate’s  Identification  Number 

Evaluator’s  Name _ 

Date  _ _ 


L-2 


DECISION  MAKING/PROBLEM  SOLVING 


Making  sound  and  timely  decisions;  paying  attention  to  and  taking  into  account  all  relevant  information  in 
making  decisions;  generating  creative  alternatives  to  solve  problems;  integrating  seemingly  unrelated 
information  and  developing  effective  solutions;  anticipating  possible  problems. 


Extremely  Effective 

a.  Extremely  difficult,  complex,  and/or  important  problem.  For  example,  limited 
information  was  available,  there  were  many  factors  to  consider  simultaneously, 
or  the  decision  had  to  be  made  immediately  and  under  pressure.  Making  the 
right  decision  was  extremely  important  (e.g.,  a  bad  decision  could  put  others’ 
health  or  safety  at  risk). 

b.  Extremely  effective  behavior.  The  candidate  collected  and  used  relevant 
information,  very  skillfully  applying  logic  to  identify  the  strengths  and 
weaknesses  of  different  approaches;  after  considering  the  options,  the 
candidate  quickly  chose  and  implemented  a  solution.  The  candidate  may  have 
created  back-up  plans  to  handle  possible  challenges. 

c.  The  solution  was  the  best  option  given  the  situation;  it  might  have  been 
especially  innovative  or  efficient. 


Moderately  Effective 

a.  Moderately  difficult/complex  problem.  For  example,  the  necessary  information 
was  available,  there  were  a  few  factors  to  consider  simultaneously,  or  the 
decision  had  to  be  made  quickly.  Making  the  right  decision  was  moderately 
important  (e.g.,  a  bad  decision  could  mean  inefficient  use  of  resources  or  a 
delay  in  completing  a  task). 

b.  Moderately  effective  behavior.  The  candidate  used  some  logic  and  relevant 
information  to  identify  the  strengths  and  weaknesses  of  different  approaches; 
may  have  hesitated  to  act  or  missed  a  slightly  better  approach.  The  candidate 
may  not  have  been  fully  prepared  for  possible  challenges. 

c.  The  solution  was  workable  given  the  situation,  though  perhaps  not  the  best;  the 
solution  might  have  been  somewhat  innovative  or  efficient. 


Ineffective 

a.  Straightforward  problem.  For  example,  there  was  a  very  obvious  solution  or 
plenty  of  time  to  react.  Making  the  right  decision  was  only  slightly  important 
(e.g.,  a  bad  decision  might  cause  inconvenience). 

b.  Ineffective  behavior.  The  candidate  ignored  or  failed  to  get  relevant  information, 
and/or  did  not  use  a  logical  process  to  examine  alternatives  or  to  make  a 
decision;  may  have  acted  too  slowly  or  not  at  all.  Candidate  might  have  chosen 
the  most  obvious  solution  without  giving  it  much  thought,  ignoring  or  failing  to 
see  the  negative  consequences  of  the  solution  picked. 

c.  The  solution  was  problematic  and/or  did  not  use  available  resources. 


L-3 


MAINTAINING  GOOD  WORKING  RELATIONSHIPS 


Developing  and  maintaining  smooth  and  effective  working  relationships  with  superiors,  peers,  and 
subordinates;  working  well  and  developing  effective  relationships  with  highly  diverse  personalities. 


Extremely  Effective 

a.  Extremely  difficult  interpersonal  situation.  Another  person  was  constantly 
behaving  in  ways  that  made  it  extremely  difficult  for  the  candidate  to  like, 
respect,  and/or  work  with  him  or  her.  The  candidate  may  have  been  faced  with 
interacting  with  many  diverse  people  (e.g.,  differing  in  rank,  position,  or 
personality).  There  might  have  been  a  major  conflict  or  dispute  between  the 
candidate  and  others. 

b.  The  candidate’s  behavior  was  extremely  effective.  The  candidate  was 
respectful  and  professional  to  the  target  person(s)  at  all  times;  the  candidate 
took  specific  actions  to  strengthen  the  relationship(s)  (for  example,  the 
candidate  might  have  introduced  him/herself,  demonstrated  openness,  listened 
carefully  to  others’  perspectives,  taken  responsibility  for  his  or  her  own  actions, 
kept  the  other  person  informed,  or  offered  support). 

c.  The  candidate’s  actions  left  the  other  person  feeling  positively  about 
him/herself  (for  example,  by  building  his  or  her  confidence)  and  the 
relationship. 


Moderately  Effective 

a.  Moderately  difficult  interpersonal  situation.  Another  person  either  did  one  thing 
that  made  him/her  difficult  to  work  with,  or  was  regularly  behaving  in  ways  that 
made  it  difficult  to  develop  a  good  relationship.  There  might  have  been  some 
difference  of  opinion  between  the  candidate  and  others. 

b.  The  candidate’s  behavior  was  moderately  effective.  Although  the  candidate 
could  have  been  more  sincere  or  more  consistent,  he/she  was  respectful  and 
professional;  the  candidate  did  not  put  the  other  person  down  or  belittle  him  or 
her. 

c.  The  candidate  and  the  other  person  might  each  deserve  some  of  the  credit  for 
maintaining  or  improving  the  relationship. 


Ineffective 

a.  Not  a  difficult  interpersonal  situation.  The  situation  clearly  specified  the  role 
expected  of  the  candidate.  The  other  person(s)  was  (were)  cooperative  and 
respectful. 

b.  The  candidate’s  behavior  was  not  effective.  The  candidate  said  or  did 
something  that  was  rude  or  unprofessional  (for  example,  the  candidate  was 
harsh,  made  the  other  person  feel  badly  about  him/herself,  or  embarrassed  or 
humiliated  the  other  person). 

c.  The  candidate’s  actions  harmed  the  relationship.  If  the  relationship  improved,  it 
was  mostly  due  to  efforts  on  the  other  person’s  part. 
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ORGANIZATIONAL  COMMITMENT 


Working  effectively  within  the  framework  of  organizational  policies,  procedures,  and  rules;  carrying  out 
orders  and  directives;  supporting  the  organization’s  mission  and  objectives. 


Extremely  Effective 

a.  Extremely  difficult  situation.  Showing  organizational  commitment  would  require 
the  candidate  to  make  a  major  personal  sacrifice  or  do  something  that 
conflicted  with  his  or  her  personal  goals,  needs  or  beliefs.  Policies,  procedures, 
or  rules  may  have  made  it  difficult  to  support  the  organization’s  mission  and 
objectives. 

b.  Strong  organizational  commitment.  The  candidate  exceeded  expectations, 
supported  the  organization’s  mission  and  objectives,  and  followed  policies  and 
procedures. 

c.  High  level  of  concern  for  success  of  the  unit/battalion.  The  candidate 
emphasized  the  importance  of  the  image  or  success  of  the  unit/battalion.  If 
applicable,  the  candidate  put  the  group’s  or  organization’s  needs  above  his  or 
her  own. 


Moderately  Effective 

a.  Moderately  difficult  situation.  Showing  organizational  commitment  would 
require  the  candidate  to  make  a  small  personal  sacrifice  or  do  something  that 
was  inconvenient  or  unpleasant.  Policies,  procedures,  or  rules  may  have  made 
it  unclear  how  best  to  support  the  organization’s  mission  and  objectives. 

b.  Moderate  organizational  commitment.  The  candidate  met  expectations  and 
complied  with  rules  or  orders. 

c.  Moderate  level  of  concern  for  success  of  the  unit/battalion.  The  candidate 
mentioned  the  importance  of  the  image  or  success  of  the  unit/battalion.  If 
applicable,  the  candidate  considered  his  own  needs  as  if  they  were  just  as 
important  as  the  needs  of  the  larger  group. 


Ineffective 

a.  Not  a  difficult  situation.  Showing  organizational  commitment  should  have  been 
easy  for  the  candidate  (e.g.,  the  task  was  a  standard  part  of  the  candidate’s 
job).  Policies,  procedures,  and  rules  may  have  facilitated  support  of  the 
organization’s  mission  and  objectives. 

b.  Weak  organizational  commitment.  The  candidate  broke  a  rule,  defied  an  order, 
or  simply  did  not  meet  expectations  (e.g.,  failed  to  complete  an  assigned  task 
on  time). 

c.  Low  level  of  concern  for  success  of  the  unit/battalion.  The  candidate  expressed 
little  or  no  concern  for  the  success  of  the  unit/battalion.  If  applicable,  the 
candidate  prioritized  his/her  own  comfort  or  convenience  above  the  greater 
good  of  the  group. 
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SHOWING  INITIATIVE 


Taking  the  initiative  to  do  all  that  is  necessary  to  accomplish  objectives  even  if  they  are  not  normally  a 
part  of  one’s  duties;  finding  additional  productive  work  to  perform  when  one’s  duties  are  completed. 


Extremely  Effective 

_  a.  High  degree  of  initiative  required.  The  work  was  clearly  not  the  candidate’s 

responsibility  (for  example,  the  task  was  above  and  beyond  the  candidate’s 
usual  duty,  or  it  was  someone  else’s  responsibility).  The  candidate  was 
completely  free  to  decide  whether  or  not  to  do  the  work  and  may  have  sought 
out  additional  responsibilities. 

_  b.  Extremely  effective  behavior.  The  candidate  welcomed  the  responsibility  and 

dedicated  more  time  and/or  effort  than  expected  to  make  sure  the  objectives 
were  met. 

_  c.  The  candidate’s  efforts  contributed  meaningfully  to  the  mission  of  the  unit. 


Moderately  Effective 

a.  Moderate  degree  of  initiative  required.  The  work  was  the  candidate’s 
responsibility  to  a  certain  extent  (for  example,  it  might  have  been  a  routine 
responsibility  that  it  would  be  beneficial  to  complete  more  quickly  than  usual,  or 
it  might  have  been  assigned  to  the  candidate’s  unit  in  general).  There  was 
some  pressure  on  the  candidate  to  accept  the  responsibility,  but  refusal  would 
have  been  acceptable.  The  candidate  may  have  agreed  to  volunteer  time  and 
effort  when  asked. 

b.  Moderately  effective  behavior.  The  candidate  did  what  was  required  to 
accomplish  objectives. 


_  c.  The  candidate’s  efforts  were  helpful  to  the  unit. 


Ineffective 

a.  Low  degree  of  initiative  required.  The  work  was  clearly  the  candidate’s 
responsibility  (for  example,  the  task  was  a  regular  part  of  the  job  or  a  superior 
ordered  the  candidate  to  do  it).  The  candidate  had  little  choice  but  to  do  the 
task. 

b.  Ineffective  behavior.  The  candidate  complained,  procrastinated,  or  put  minimal 
effort  into  accomplishing  objectives. 


_  c.  The  candidate’s  effort  (or  lack  of  effort)  was  unhelpful  or  hindered  the  unit. 
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ADAPTING  TO  CHANGING  OR  UNCERTAIN  SITUATIONS 


Easily  changing  gears  in  response  to  unpredictable  or  unexpected  events  and  circumstances;  effectively 
adjusting  plans,  goals,  actions,  or  priorities  to  deal  with  changing  situations;  imposing  structure  for  self 
and  others  that  provides  as  much  focus  as  possible  in  dynamic  situations;  not  needing  things  to  be  black 
and  white;  refusing  to  be  paralyzed  by  uncertainty  or  ambiguity. 


‘K 

u/ 


Extremely  Effective 

a.  Extremely  difficult  situation  involving  substantial,  abrupt,  or  constant  changes  or 
sustained  uncertainty;  the  response  may  have  had  to  be  made  immediately  and 
under  strong  pressure.  Failure  to  adapt  quickly  could  have  severe  negative 
consequences,  such  as  endangering  others’  safety  or  substantial 
inconvenience  to  many  other  people. 

b.  The  candidate  reacted  quickly  and  willingly  to  account  for  the  changes,  without 
unnecessary  hesitation  or  debate. 

c.  The  candidate  reacted  calmly  and  confidently  to  the  circumstances.  The 
candidate  may  have  shown  leadership  by  providing  structure  and/or  a  good 
example  to  others  involved. 


b. 


c. 


Moderately  Effective 

Moderately  difficult  situation  involving  a  moderate  degree  of  change,  a  series  of 
minor  changes  in  a  short  time,  or  some  uncertainty;  the  response  may  have 
had  to  be  made  quickly  and  under  some  pressure.  Failure  to  adapt  could  have 
some  negative  consequences 

Although  the  candidate  did  adapt  before  it  was  too  late,  he  or  she  initially 
hesitated  or  resisted  the  change. 

The  candidate  adapted  well  enough  to  address  the  change  and  accomplish 
something  productive  without  creating  negative  outcomes. 


n/ 


Ineffective 

_  a.  The  situation  was  not  difficult,  involving  minor  or  gradual  changes  for  the 

candidate.  Failure  to  adapt  could  have  minor  consequences,  such  as  a  small 
inconvenience  to  the  candidate  or  a  few  other  people. 

_  b.  The  candidate  adapted  very  slowly,  very  unwillingly,  or  not  at  all. 

_  c.  The  candidate’s  response  to  the  circumstances  created  a  negative  outcome 

(for  example,  others  were  endangered,  hindered,  inconvenienced,  or 
demotivated). 
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Past  Behavior  Record: 
Evaluation  Guide 


VERSION:  M  14-Sep-06 


Prepared  for: 

U.S.  Army  Research  Institute 


Candidate’s  Identification  Number 

Evaluator’s  Name _ 

Date  _ 
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DECISION  MAKING/PROBLEM  SOLVING 


Making  sound  and  timely  decisions;  paying  attention  to  and  taking  into  account  all  relevant 
information  in  making  decisions;  generating  creative  alternatives  to  solve  problems;  integrating 
seemingly  unrelated  information  and  developing  effective  solutions;  anticipating  possible 
problems. 

Rating  Instructions:  Please  mark  one  option  in  Part  A,  one  in  Part  B,  and  one  in  Part  C. 

Part  A.  Difficulty  or  importance  of  the  decision  or  problem: 

_ 1 .  Extremely  difficult  and/or  important.  The  problem  was  highly  complex;  for  example, 

limited  information  was  available,  there  were  many  factors  to  consider 
simultaneously,  or  the  decision  had  to  be  made  immediately  and  under  pressure. 
Making  the  right  decision  was  extremely  important  (e.g.,  a  bad  decision  could  put 
others’  health  or  safety  at  risk). 

_ 2.  Moderately  difficult.  The  problem  was  moderately  complex;  for  example,  the 

necessary  information  was  available,  there  were  a  few  factors  to  consider 
simultaneously,  or  the  decision  had  to  be  made  quickly.  Making  the  right  decision 
was  moderately  important  (e.g.,  a  bad  decision  could  mean  inefficient  use  of 
resources  or  a  delay  in  completing  a  task). 

_ 3.  Not  difficult.  The  problem  was  straightforward;  for  example,  there  was  a  very 

obvious  solution  or  plenty  of  time  to  react.  Making  the  right  decision  was  only 
slightly  important  (e.g.,  a  bad  decision  might  cause  inconvenience). 

Part  B.  Effectiveness  of  the  candidate’s  behavior: 

_ 1.  Extremely  effective.  The  candidate  collected  and  used  relevant  information,  very 

skillfully  applying  logic  to  identify  the  strengths  and  weaknesses  of  different 
approaches;  after  considering  the  options,  the  candidate  quickly  chose  and 
implemented  a  solution.  The  candidate  may  have  created  back-up  plans  to  handle 
possible  challenges. 

_ 2.  Moderately  effective.  The  candidate  used  some  logic  and  relevant  information  to 

identify  the  strengths  and  weaknesses  of  different  approaches;  may  have  hesitated 
to  act  or  missed  a  slightly  better  approach.  The  candidate  may  not  have  been  fully 
prepared  for  possible  challenges. 

_ 3.  Not  effective.  The  candidate  ignored  or  failed  to  get  relevant  information,  and/or  did 

not  use  a  logical  process  to  examine  alternatives  or  to  make  a  decision;  may  have 
acted  too  slowly  or  not  at  all.  Candidate  might  have  chosen  the  most  obvious 
solution  without  giving  it  much  thought,  ignoring  or  failing  to  see  the  negative 
consequences  of  the  solution  picked. 

Part  C.  Quality  of  the  outcome: 

_ 1.  The  solution  was  the  best  option  given  the  situation;  it  might  have  been  especially 

innovative  or  efficient. 

_ 2.  The  solution  was  workable  given  the  situation,  though  perhaps  not  the  best;  the 

solution  might  have  been  somewhat  innovative  or  efficient. 

_ 3.  The  solution  was  problematic  and/or  did  not  use  available  resources. 


M-3 


MAINTAINING  GOOD  WORKING  RELATIONSHIPS 


Developing  and  maintaining  smooth  and  effective  working  relationships  with  superiors,  peers, 
and  subordinates;  working  well  and  developing  effective  relationships  with  highly  diverse 
personalities. 

Rating  Instructions:  Please  mark  one  option  in  Part  A,  one  in  Part  B,  and  one  in  Part  C. 

Part  A.  Difficulty  of  the  interpersonal  situation: 

_ 1 .  Extremely  difficult.  Another  person  was  constantly  behaving  in  ways  that  made  it 

extremely  difficult  for  the  candidate  to  like,  respect,  and/or  work  with  him  or  her. 
The  candidate  may  have  been  faced  with  interacting  with  many  diverse  people 
(e.g.,  differing  in  rank,  position,  or  personality).  There  might  have  been  a  major 
conflict  or  dispute  between  the  candidate  and  others. 

_ 2.  Moderately  difficult.  Another  person  either  did  one  thing  that  made  him/her  difficult 

to  work  with,  or  was  regularly  behaving  in  ways  that  made  it  difficult  to  develop  a 
good  relationship.  There  might  have  been  some  difference  of  opinion  between  the 
candidate  and  others. 

_ 3.  Not  difficult.  The  situation  clearly  specified  the  role  expected  of  the  candidate.  The 

other  person(s)  was  (were)  cooperative  and  respectful. 

Part  B.  Effectiveness  of  the  candidate’s  behavior  (be  sure  to  focus  on  what  the  candidate  did, 
not  what  others  in  the  situation  did): 

_ 1 .  Extremely  effective.  The  candidate  was  respectful  and  professional  to  the  target 

person(s)  at  all  times;  the  candidate  took  specific  actions  to  strengthen  the 
relationship(s)  (for  example,  the  candidate  might  have  introduced  him/herself, 
demonstrated  openness,  listened  carefully  to  others’  perspectives,  taken 
responsibility  for  his  or  her  own  actions,  kept  the  other  person  informed,  or  offered 
support). 

_ 2.  Moderately  effective.  Although  the  candidate  could  have  been  more  sincere  or 

more  consistent,  he/she  was  respectful  and  professional;  the  candidate  did  not  put 
the  other  person  down  or  belittle  him  or  her. 

_ 3.  Not  effective.  The  candidate  said  or  did  something  that  was  rude  or  unprofessional 

(for  example,  the  candidate  was  harsh,  made  the  other  person  feel  badly  about 
him/herself,  or  embarrassed  or  humiliated  the  other  person). 

Part  C.  Impact  of  the  candidate’s  behavior: 

_ 1 .  The  candidate’s  actions  left  the  other  person  feeling  positively  about  him/herself 

(for  example,  by  building  his  or  her  confidence)  and  the  relationship. 

_ 2.  The  candidate  and  the  other  person  might  each  deserve  some  of  the  credit  for 

maintaining  or  improving  the  relationship. 

_ 3.  The  candidate’s  actions  harmed  the  relationship.  If  the  relationship  improved,  it 

was  mostly  due  to  efforts  on  the  other  person’s  part. 
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ORGANIZATIONAL  COMMITMENT 


Working  effectively  within  the  framework  of  organizational  policies,  procedures,  and  rules; 

carrying  out  orders  and  directives;  supporting  the  organization’s  mission  and  objectives. 

Rating  Instructions:  Please  mark  one  option  in  Part  A,  one  in  Part  B,  and  one  in  Part  C. 

Part  A.  Difficulty  of  the  situation: 

_ 1.  Extremely  difficult.  Showing  organizational  commitment  would  require  the 

candidate  to  make  a  major  personal  sacrifice  or  do  something  that  conflicted  with 
his  or  her  personal  goals,  needs  or  beliefs.  Policies,  procedures,  or  rules  may  have 
made  it  difficult  to  support  the  organization’s  mission  and  objectives. 

_ 2.  Moderately  difficult.  Showing  organizational  commitment  would  require  the 

candidate  to  make  a  small  personal  sacrifice  or  do  something  that  was 
inconvenient  or  unpleasant.  Policies,  procedures,  or  rules  may  have  made  it 
unclear  how  best  to  support  the  organization’s  mission  and  objectives. 

_ 3.  Not  difficult.  Showing  organizational  commitment  should  have  been  easy  for  the 

candidate  (e.g.,  the  task  was  a  standard  part  of  the  candidate’s  job).  Policies, 
procedures,  and  rules  may  have  facilitated  support  of  the  organization’s  mission 
and  objectives. 

Part  B.  Strength  of  the  candidate’s  organizational  commitment: 

_ 1.  Strong.  The  candidate  exceeded  expectations,  supported  the  organization’s 

mission  and  objectives,  and  followed  policies  and  procedures. 

_ 2.  Moderate.  The  candidate  met  expectations  and  complied  with  rules  or  orders. 

_ 3.  Weak.  The  candidate  broke  a  rule,  defied  an  order,  or  simply  did  not  meet 

expectations  (e.g.,  failed  to  complete  an  assigned  task  on  time). 

Part  C.  Level  of  the  candidate’s  concern  for  success  of  the  unit/battalion: 

_ 1 .  High.  The  candidate  emphasized  the  importance  of  the  image  or  success  of  the 

unit/battalion.  If  applicable,  the  candidate  put  the  group’s  or  organization’s  needs 

above  his  or  her  own. 

* 

_ 2.  Moderate.  The  candidate  mentioned  the  importance  of  the  image  or  success  of  the 

unit/battalion.  If  applicable,  the  candidate  considered  his  own  needs  as  if  they  were 
just  as  important  as  the  needs  of  the  larger  group. 

_ 3.  Low.  The  candidate  expressed  little  or  no  concern  for  the  success  of  the 

unit/battalion.  If  applicable,  the  candidate  prioritized  his/her  own  comfort  or 
convenience  above  the  greater  good  of  the  group. 
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SHOWING  INITIATIVE 


Taking  the  initiative  to  do  all  that  is  necessary  to  accomplish  objectives  even  if  they  are  not 
normally  a  part  of  one’s  duties;  finding  additional  productive  work  to  perform  when  one’s  duties 
are  completed. 

Rating  Instructions:  Please  mark  one  option  in  Part  A,  one  in  Part  B,  and  one  in  Part  C. 

Part  A.  Degree  of  initiative  required: 

_ 1.  High.  The  work  was  clearly  not  the  candidate’s  responsibility  (for  example,  the  task 

was  above  and  beyond  the  candidate’s  usual  duty,  or  it  was  someone  else’s 
responsibility).  The  candidate  was  completely  free  to  decide  whether  or  not  to  do 
the  work  and  may  have  sought  out  additional  responsibilities. 

_ 2.  Moderate.  The  work  was  the  candidate’s  responsibility  to  a  certain  extent  (for 

example,  it  might  have  been  a  routine  responsibility  that  it  would  be  beneficial  to 
complete  more  quickly  than  usual,  or  it  might  have  been  assigned  to  the 
candidate’s  unit  in  general).  There  was  some  pressure  on  the  candidate  to  accept 
the  responsibility,  but  refusal  would  have  been  acceptable.  The  candidate  may 
have  agreed  to  volunteer  time  and  effort  when  asked. 

_ 3.  Low.  The  work  was  clearly  the  candidate’s  responsibility  (for  example,  the  task  was 

a  regular  part  of  the  job  or  a  superior  ordered  the  candidate  to  do  it).  The  candidate 
had  little  choice  but  to  do  the  task. 

Part  B.  Effectiveness  of  the  candidate’s  behavior: 

_ 1.  Extremely  effective.  The  candidate  welcomed  the  responsibility  and  dedicated 

more  time  and/or  effort  than  expected  to  make  sure  the  objectives  were  met. 

_ 2.  Moderately  effective.  The  candidate  did  what  was  required  to  accomplish 

objectives. 

_ 3.  Not  effective.  The  candidate  complained,  procrastinated,  or  put  minimal  effort  into 

accomplishing  objectives. 

Part  C.  Impact  of  the  candidate’s  behavior: 

_ 1.  The  candidate’s  efforts  contributed  meaningfully  to  the  mission  of  the  unit. 

_ _  2.  The  candidate’s  efforts  were  helpful  to  the  unit. 

_ 3.  The  candidate’s  effort  (or  lack  of  effort)  was  unhelpful  or  hindered  the  unit. 
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ADAPTING  TO  CHANGING  OR  UNCERTAIN  SITUATIONS 


Easily  changing  gears  in  response  to  unpredictable  or  unexpected  events  and  circumstances; 
effectively  adjusting  plans,  goals,  actions,  or  priorities  to  deal  with  changing  situations;  imposing 
structure  for  self  and  others  that  provides  as  much  focus  as  possible  in  dynamic  situations;  not 
needing  things  to  be  black  and  white;  refusing  to  be  paralyzed  by  uncertainty  or  ambiguity. 

Rating  Instructions:  Please  mark  one  option  in  Part  A,  one  in  Part  B,  and  one  in  Part  C. 

Part  A.  Difficulty  of  the  situation: 

_ 1.  Extremely  difficult.  The  situation  involved  substantial,  abrupt,  or  constant  changes 

or  sustained  uncertainty;  the  response  may  have  had  to  be  made  immediately  and 
under  strong  pressure.  Failure  to  adapt  quickly  could  have  severe  negative 
consequences,  such  as  endangering  others’  safety  or  substantial  inconvenience  to 
many  other  people. 

_ 2.  Moderately  difficult.  The  situation  involved  a  moderate  degree  of  change,  a  series 

of  minor  changes  in  a  short  time,  or  some  uncertainty;  the  response  may  have  had 
to  be  made  quickly  and  under  some  pressure.  Failure  to  adapt  could  have  some 
negative  consequences 

_ 3.  Not  difficult.  The  situation  involved  minor  or  gradual  changes  for  the  candidate. 

Failure  to  adapt  could  have  minor  consequences,  such  as  a  small  inconvenience 
to  the  candidate  or  a  few  other  people. 

Part  B.  Readiness  of  the  candidate  to  adapt: 

_ 1 .  The  candidate  reacted  quickly  and  willingly  to  account  for  the  changes,  without 

unnecessary  hesitation  or  debate. 

_ 2.  Although  the  candidate  did  adapt  before  it  was  too  late,  he  or  she  initially  hesitated 

or  resisted  the  change. 

_ 3.  The  candidate  adapted  very  slowly,  very  unwillingly,  or  not  at  all. 

Part  C.  Effectiveness  of  the  candidate’s  adaptive  behavior: 

_ 1.  Extremely  effective.  The  candidate  reacted  calmly  and  confidently  to  the 

circumstances.  The  candidate  may  have  shown  leadership  by  providing  structure 
and/or  a  good  example  to  others  involved. 

_ 2.  Moderately  effective.  The  candidate  adapted  well  enough  to  address  the  change 

and  accomplish  something  productive  without  creating  negative  outcomes. 

_ 3.  Not  effective.  The  candidate’s  response  to  the  circumstances  created  a  negative 

outcome  (for  example,  others  were  endangered,  hindered,  inconvenienced,  or 
demotivated). 
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Intercorrelations  Between  Predictor  Variables  Relevant  to 
All  Performance  Dimensions 
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Table  N 

Intercorrelations  Between  Predictor  V ariables  Common  to  All  Performance  Dimensions 
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Note.  Pairwise  deletion  of  missing  cases.  Ns  range  from  77  to  155.  *p  <  .05.  "p  <  .01.  ***p  <  .001. 
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24.  Assertiveness 
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